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ABSTRACT 


This  thesis  examined  the  evolution  of  the  Direct  Reporting  Program  Manager- 
Advanced  Amphibious  Assault’s  test  and  evaluation  strategy  from  Milestone  0  to  the 
present.  The  research  effort  involved  reviewing  the  evolution  of  amphibious  doctrine  and 
amphibious  vehicles,  reviewing  the  DoD  Acquisition  Process  and  the  role  of  T&E  in  that 
Acquisition  Process,  and  analyzing  three  DRPM-AAA  Test  and  Evaluation  Master  Plans. 
Interviews  were  conducted  with  personnel  from  the  DRPM-AAA  office  and  General 
Dynamics  Amphibious  Systems.  Additionally,  program  documents  and  acquisition 
literature  were  reviewed.  An  analysis  of  test  and  evaluation  issues  facing  the  Program 
Management  Office,  a  determination  of  the  effects  those  issues  had  on  the  program’s  test 
strategy,  and  applicable  lessons  learned  are  documented  for  use  by  other  major  defense 
acquisition  programs.  Key  research  findings  conclude:  that  the  DRPM-AAA’ s  T&E 
Strategy  remained  stable  and  consistent  from  Milestone  0  to  the  present  as  a  result  of  the 
continuity  of  the  AAAV’s  Key  Performance  Parameters;  that  the  DRPM’s  decision  to 
develop  a  working  relationship  that  “actively  engages”  both  oversight  and  external 
agencies  early  in  the  test  planning  process  serves  in  achieving  test  resource  efficiencies; 
and  that  the  IPT  decision-making  process  differs  significantly  from  the  more  formal  “staff 
planning  process”  used  by  most  military  organizations. 
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I.  INTRODUCTION 


A.  PURPOSE 

The  purpose  of  this  research  is  to  examine  the  evolution  of  the  AAAV  Program 
Management  Office’s  test  and  evaluation  strategy  from  Milestone  0  to  the  present.  The 
goal  is  to  analyze  the  critical  test  and  evaluation  issues  facing  the  Program  Management 
Office,  analyze  the  effect  of  those  issues  on  the  program’s  test  strategy,  and  to  develop 
applicable  lessons  learned  for  other  major  defense  acquisition  programs. 

B.  BACKGROUND 

Recent  changes  in  the  focus  of  naval  and  amphibious  warfare  have  resulted  in  a 
need  for  both  the  Navy  and  Marine  Corps  to  conduct  their  operations  from  over-the- 
horizon  (OTH).  This  evolution  in  amphibious  warfare  doctrine  is  based  on  a  triad  of 
surface  and  air  delivery  platforms  capable  of  getting  Marines  and  their  equipment  from 
ship-to-shore  in  a  more  rapid  and  less  vulnerable  way  [Ref.  1].  Existing  Marine  Corps 
Assault  Amphibian  Vehicles  (AAVs)  do  not  meet  the  needs  of  this  developing  over-the- 
horizon  doctrine.  [Ref.  2:p.  6]  An  Advanced  Amphibious  Assault  Vehicle  program  to 
meet  this  need  was  initiated,  canceled,  and  restarted  several  times  in  the  mid  and  late 
1980's  before  a  Program  Management  Office  was  finally  established  in  June  of  1990. 
[Ref.  3:p.  2] 

As  the  primary  ground  combat  acquisition  program  in  the  Marine  Corps,  the 
AAAV  faces  much  scrutiny  from  different  Department  of  Defense  (DoD)  organizations 
and  the  United  States  Congress.  In  fact,  Congressional  involvement  in  major  defense 
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acquisition  programs  (MDAPs)  has  increased  significantly  over  the  past  decade  as  they 
have  sought  ways  to  improve  the  effectiveness  of  the  DoD  Acquisition  Process.  [Ref.  4:p. 
21]  One  area  of  the  acquisition  process  that  is  probably  the  most  visible,  and  therefore 
receives  the  greatest  scrutiny  is  the  test  and  evaluation  process  of  a  major  defense 
acquisition  program.  Congress  has  mandated  that  it  receive  a  Live-Fire  Test  and 
Evaluation  Report  and  a  Beyond  Low-Rate  Initial  Production  (LRIP)  Report  indicating  a 
program's  readiness  to  proceed  into  full  rate  production  prior  to  approval  to  do  so.  [Ref. 
5:part  3.4]  In  order  for  a  Program  Manager  to  successfully  develop  a  system  capable  of 
meeting  the  user's  operational  requirements,  he  must  establish  a  test  and  evaluation 
strategy  that  tracks  the  system's  capabilities  and  performance  throughout  the  entire 
acquisition  cycle. 

Previous  studies  of  the  AAAV  PMO  have  indicated  that  it  has  successfully  made 
critical  decisions  concerning  acquisition  reform  measures  to  reduce  the  risk  of  cost  and 
schedule  overruns,  and  ineffective  performance.  [Ref.  3,  6,  7]  These  studies  also  indicate 
that  the  PMO  was  forward  thinking  in  its  overall  acquisition  strategy  by  streamlining  the 
contractor  /  government  relationship,  increasing  the  use  of  Integrated  Product  Process 
Development  (IPPD)  and  Integrated  Product  Teams  (IPTs),  and  adopting  commercial 
business  practices.  [Ref.  3]  One  study  in  particular  praised  the  PMO  for  its  risk 
management  and  mitigation  methodology,  yet  indirectly  questions  the  PMO  for  its  use  of 
a  "classical"  test  and  evaluation  approach  in  an  otherwise  "evolutionary"  acquisition 
strategy.  The  study  recommended  further  research  into  the  issues,  feasibility,  advantages, 
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and  disadvantages  associated  with  a  "classical"  T&E  approach  in  an  “evolutionary” 
acquisition  strategy.  [Ref.  6:p.  79] 

This  thesis,  therefore,  will  examine  the  evolution  of  the  AAAV  Program 
Management  Office’s  test  and  evaluation  strategy  from  Milestone  0  to  the  present.  The 
goal  is  to  analyze  the  critical  test  and  evaluation  issues  facing  the  Program  Management 
Office,  analyze  the  effect  of  those  issues  on  the  program’s  test  strategy,  and  to  develop 
applicable  lessons  for  other  major  defense  acquisition  programs. 

C.  RESEARCH  QUESTIONS 

1.  Primary 

The  primary  research  questions  that  this  thesis  will  address  are:  How  has  the 
AAAV  test  and  evaluation  strategy  evolved  from  Milestone  0  to  the  present,  and  what 
lessons  learned  can  be  derived  from  this  evolution? 

2.  Subsidiary 

The  subsidiary  research  questions  are  as  follows: 

•  What  test  and  evaluation  related  issues  have  been  faced,  and  what  decisions 
have  been  made  with  respect  to  AAAV  test  and  evaluation? 

•  What  is  the  AAAV  Developmental  Test  and  Evaluation  (DT&E)  strategy? 

•  What  is  the  AAAV  Operational  Test  and  Evaluation  (OT&E)  strategy? 

•  What  impact  has  the  implementation  of  Acquisition  Reform  measures  had  on 
the  AAAV  test  and  evaluation  process? 

•  Will  an  analysis  of  AAAV  T&E  strategy  be  useful  in  the  development  of 
future  MDAP  T&E  strategies? 
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D.  SCOPE  AND  LIMITATIONS 


The  scope  of  this  thesis  will  be  a  case  study  of  the  evolution  of  the  AAAV  PMO’s 
test  and  evaluation  strategy.  The  study  will  review  standard  test  and  evaluation  practices; 
the  contractor’s  test  and  evaluation  practices;  and,  the  test  and  evaluation  practices  as 
developed  by  the  AAAV  PMO.  The  study  will  then  analyze  specific  test  and  evaluation 
issues  effecting  the  PMO;  review  the  decisions  made  and  strategy  developed  by  the 
PMO;  and  finally,  develop  lessons  learned  that  may  be  applicable  to  other  major  defense 
acquisition  programs. 

The  study  will  not  conduct  a  critique  or  evaluation  of  the  overall  acquisition 
strategy,  nor  an  analysis  of  detailed  test  plans  for  individual  components  of  the  AAAV 
(turret,  software,  engine,  etc.). 

E.  METHODOLOGY 

The  methodology  used  in  this  thesis  research  consists  of  the  following:  (1) 
conduct  a  literature  search  of  books,  magazine  articles,  CD-ROM  systems,  and  other 
library  information  resources,  (2)  conduct  a  review  of  all  program  test  and  evaluation 
related  documents,  (3)  interview  personnel  from  the  PMO,  General  Dynamic  Amphibious 
Systems  (GDAMS),  Marine  Corps  Operational  Test  and  Evaluation  Activity  (MCOTEA), 
and  Marine  Corps  Systems  Command  (MARCORSYSCOM),  and  (4)  conduct  a  site  visit 
to  the  PMO/GDAMS  facility  to  discuss  test  and  evaluation  issues. 
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F.  ORGANIZATION  OF  STUDY 


Chapter  I.  Introduction.  Identifies  the  focus  and  purpose  of  the  thesis  and  states 
the  primary  and  subsidiary  research  questions. 

Chapter  II.  Assault  Amphibian  History.  This  chapter  provides  the  reader  with 
an  overview  of  assault  amphibian  doctrine  and  history. 

Chapter  III.  Department  of  Defense  Acquisition  Overview.  This  chapter 
provides  the  reader  with  an  extensive  overview  of  Department  of  Defense  Acquisition 
Process. 

Chapter  IV.  Department  of  Defense  Test  and  Evaluation  Overview.  This 
chapter  provides  an  overview  of  the  types  of  test  and  evaluation  in  the  DoD,  the  test  and 
evaluation  process  as  it  relates  to  the  acquisition  process,  and  an  overview  of  previous 
lessons  learned  from  both  a  developmental  and  operational  test  perspective. 

Chapter  V.  AAAV  Test  and  Evaluation  Strategy.  This  chapter  outlines  the 
PMO’s  acquisition  strategy,  test  and  evaluation  strategy,  and  the  process  used  to  reconcile 
the  established  operational  requirements  necessary  for  testing  purposes. 

Chapter  VI.  Test  and  Evaluation  Related  Issues  and  Lessons  Learned.  This 
chapter  analyzes  the  test  and  evaluation  related  issues  facing  the  Program  Management 
Office,  assesses  the  effect  of  those  issues  on  the  program’s  test  strategy,  and  develops 
applicable  lessons  for  other  major  defense  acquisition  programs 
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Chapter  VII.  Conclusions  and  Recommendations.  The  final  chapter  summarizes 
the  findings  of  the  research,  summarizes  the  lessons  learned,  provides  recommendations 
from  this  study,  and  provides  recommendations  for  future  research. 

G.  CHAPTER  SUMMARY 

This  chapter  has  identified  the  focus  and  purpose  of  the  thesis,  stated  the  primary 
and  subsidiary  research  questions,  addressed  the  scope  of  the  thesis,  and  familiarized  the 
reader  with  the  organization  of  the  study.  The  next  three  chapters  will  provide  the  reader 
the  necessary  background  to  place  the  remainder  of  the  thesis  and  its  analysis  in  context. 
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II.  ASSAULT  AMPHIBIAN  HISTORY 


A.  INTRODUCTION 

The  United  States  Marine  Corps'  quest  for  an  Advanced  Amphibious  Assault 
Vehicle  is  a  classic  example  of  the  dynamic  relationship  between  the  military  operating 
environment,  doctrine,  and  technology.  The  relationship  between  the  three  is  generally 
cyclic  and  evolutionary.  As  a  change  in  technology  occurs,  one  must  modify  or  develop 
a  new  doctrine  to  successfully  apply  that  technology.  This  new  doctrine  then  creates  a 
new  environment  in  which  the  military  must  operate.  The  AAAV  is  the  forth  generation 
of  amphibian  vehicles  resulting  from  the  Marine  Corps’  application  of  this  environment- 
doctrine-technology  cycle. 

This  chapter  examines  the  evolution  of  amphibious  doctrine  from  its  founding  in 
the  early  1930’s,  to  the  development  of  today’s  Over-The-Horizon  concepts.  The  chapter 
will  then  examine  the  evolution  of  the  first  three  generations  of  amphibian  vehicles 
designed  to  meet  changing  Marine  Corps  doctrine  with  improved  technology. 

B.  AMPHIBIOUS  DOCTRINE 

As  an  element  of  the  Naval  Service,  the  Marine  Corps  has  a  tradition  of  operating 
at  sea  and  ashore,  thus  its  amphibious  nature.  Therefore,  an  amphibious  operation  can  be 
defined  as,  "an  attack  launched  from  the  sea  by  naval  and  landing  forces,  embarked  in 
ships  or  craft  involving  a  landing  on  a  hostile  or  potentially  hostile  shore."  [Ref.  8] 

i 

Amphibious  doctrine  first  evolved  in  the  early  1920's  and  1930's  when  the  United 
States  realized  that  there  was  a  strong  likely-hood  of  a  conflict  with  Japan  in  the  Pacific. 
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The  Marine  Corps  worked  extensively  on  developing  amphibious  landing  techniques  that 

would  meet  the  challenges  of  an  island-hopping  campaign  designed  to  secure  advanced 

naval  and  air  bases  in  that  region.  In  1934,  the  Tentative  Manual  for  Landing  Operations 

(LFM  0-1)  was  published.  [Ref.  9:p.  34]  Though  refined  in  its  detail  from  experiences 

gained  in  World  War  II,  Richard  S.  Moore  stated  in  1985  that,  "[The]  current  amphibious 

doctrine,  embodied  in  LFM  0-1  ...reflects  the  past  50  years  of  development.  It  differs 

little  from  the  1934  manual."  [Ref.  9:p.  34]  Moore  questioned  the  viability  of  existing 

doctrine  in  the  face  of  changing  threat  technology. 

The  proliferation  of  missile  weapons  and  the  potential  for  the  smallest  of 
armies  to  obtain  nuclear  weapons  increase  the  vulnerability  of  an 
amphibious  landing.  That  landing's  survivability  rests  with  the  doctrine 
with  which  it  is  executed.  [Ref.  9:p.  32] 

Moore  concludes  that,  "If  future  amphibious  landings  are  to  avoid  disaster,  there  must  be 
a  resurgence  in  the  doctrinal  thought  of  the  1930's."  [Ref.  9:p.  36] 

In  1986,  the  Commandant  of  the  Marine  Corps,  General  P.  X.  Kelley,  published 
his  vision  of  Amphibious  Warfare  Strategy  as  it  related  to  the  National  Military  Strategy. 
General  Kelley  has  three  main  points  to  his  strategy.  First,  he  reaffirmed  the  Marine 
Corps'  responsibility  as  the  lead  service  in  developing  amphibious  tactics,  techniques,  and 
equipment  as  outlined  in  the  National  Security  Act  of  1947.  [Ref.  10:p.  23] 

Second,  General  Kelley  describes  the  world  environment  in  which  the  Marine 
Corps  will  operate  over  the  next  decade.  This  environment  includes  a  reduction  in 
overseas  basing  that  increases  the  significance  of  amphibious  forces.  Specifically, 
"Amphibious  forces,  on  the  other  hand,  can  be  stationed  over  the  horizon  at  sea,  need  no 
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basing  or  overflight  clearances,  and  provide  their  own  sustainment."  [Ref.  10:p.  25]  By 
defining  the  world  environment  as  such,  and  by  defining  the  role  of  amphibious  forces, 
General  Kelley  ensured  the  importance  of  an  amphibious  forcible  entry  capability  in  the 
overall  national  strategy. 

General  Kelley's  final  point  is  that  the  Navy  and  Marine  Corps  must  act  as  an 
amphibious  team  to  maximize  each  other's  military  capabilities.  Two  of  the  pillars  of  this 
teamwork  concept  are  equipment  and  doctrine.  '  With  respect  to  equipment,  General 
Kelley  identifies  the  acquisition  of  the  Landing  Craft  Air  Cushion  (LCAC)  and  the  MV- 
22  tilt-rotor  aircraft  (Osprey)  as  significant.  He  says,  "These  platforms  will  allow  for  a 
much  more  rapid  closure  to  the  beach,  giving  the  amphibious  task  force  the  option  of 
operating  from  over  the  horizon,  out  of  range  of  many  enemy  weapon  systems."  [Ref. 
10:p.  27-29]  With  respect  to  doctrine,  General  Kelley  concludes,  "We  believe  that  there 
is  ample  evidence  to  suggest  that  we  have  entered  a  renaissance  period  in  the  evolution  of 
amphibious  operations,  with  the  broadening  of  our  vistas  through  the  introduction  of  the 
LCAC  and  Osprey  yet  to  come."  [Ref.  10:p.  29] 

•  The  demise  of  the  Soviet  Union  and  the  end  of  the  Cold  War  brought  about  a 
renaissance  in  not  only  amphibious  strategy,  but  also  the  overall  maritime  strategy.  In 
September  1992,  the  Secretary  of  the  Navy,  Chief  of  Naval  Operations,  and  the 
Commandant  of  the  Marine  Corps  published  their  vision  of  how  to  meet  this  significant 
shift  in  the  military  operating  environment.  The  White  Paper,  "...From  the  Sea:  A  New 
Direction  for  the  Naval  Services",  identifies  this  new  operating  environment,  explains  the 
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naval  services  role  in  this  environment,  and  assigns  tasks  to  carry  out  that  role.  [Ref. 

1 1  :pp.  18-22] 

The  new  direction  for  the  naval  services  "represents  a  significant  shift  away  from 
open-ocean  warfighting  on  the  sea  toward  joint  operations  conducted  from  the  sea."  [Ref. 
1 1  :p.  19]  Operating  from  the  sea  means  operating  in  the  near  land,  or  littoral,  areas  of  the 
world  in  an  environment  that  poses  varying  technical  and  tactical  challenges  to  naval 
forces.  [Ref.  1 1  :p.  20]  The  naval  services'  role  in  the  littoral  region  will  be  met  by 
providing  strong  naval  expeditionary  forces,  specifically  tailored  and  capable  of 
projecting  power  ashore. 

The  white  paper  assigns  15  tasks  to  the  Navy  and  Marine  Corps  in  order  to 
achieve  this  new  vision  of  maneuvering  from  the  sea.  Of  the  15,  the  following  three 
indirectly  contribute  to  the  Marine  Corps'  justification  for  the  AAAV  [Ref.  1 1  :p.  22]: 

•  Develop  naval  doctrine  consistent  with  new  direction  and  focus- 
including  an  examination  of  functions  and  capabilities. 

•  Increase  emphasis  on  generation  of  high-intensity  power  projection, 
support  of  forces  ashore,  and  weapons  necessary  to  fulfill  the  mission. 

•  Procure  equipment  systems  to  support  this  strategy  and  remain  ahead 
of  the  global  technology  revolution  in  military  systems. 

In  1994,  the  Secretary  of  the  Navy,  Chief  of  Naval  Operations,  and  the 
Commandant  of  the  Marine  Corps  published  "Forward... From  the  Sea."  This  white 
paper,  "updates  and  expands  the  strategic  concept  to  address  specifically  the  unique 
contributions  of  naval  expeditionary  forces  in  peacetime  operations,  responding  to  crises, 
and  in  regional  conflicts."  [Ref.  12:pp.  32-35]  "Forward... From  the  Sea"  reaffirms  the 
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direction  of  evolving  strategy,  confirms  the  need  for  naval  expeditionary  forces,  and 
establishes  projection  of  power  from  sea  to  land  as  the  number  one  prevailing  role  for 
naval  forces.  [Ref.  12:p.  35] 

On  4  January  1996,  the  Commandant  of  the  Marine  Corps,  General  C.  C.  Krulak, 
approved  the  Marine  Corps'  contribution  to  this  evolving  doctrine  by  releasing 
"Operational  Maneuver  From  the  Sea:  A  Concept  for  the  Projection  of  Naval  Power 
Ashore."  [Ref.  13:p.  A1-A6]  General  Krulak  indicates  that  Operational  Maneuver  from 
the  Sea  (OMFTS)  results  from  two  changes  in  the  operational  environment.  First,  a  new 
series  of  threats  is  resulting  from  national  aspirations,  religious  intolerance,  and  ethnic 
hatred  in  the  littoral  region.  Second,  opportunities  from  enhancements  in  information 
management,  battlefield  mobility,  and  conventional  weapons'  lethality  will  allow  for 
increased  Marine  Corps  capabilities.  [Ref.  13:p.  A-l] 

The  concept  paper  states: 

What  distinguishes  OMFTS  from  all  other  species  of  operational 
maneuver  is  the  extensive  use  of  the  sea  as  a  means  of  gaining  advantage, 
an  avenue  for  friendly  movement  that  is  simultaneously  a  barrier  to  the 
enemy  and  a  means  of  avoiding  disadvantageous  engagements.  This 
aspect  of  OMFTS  may  make  use  of.  ..the  sea  as  a  medium  for  tactical  and 
operational  movement.  [Ref.  13:p.  A-3] 

The  concept  paper  further  asserts  that,  "landing  forces  armed  with  the  tactical 
mobility  and  fire  support  capabilities  of  the  present  will  be  hard  pressed  to  decisively 
engage  an  enemy  who  is  likely  to  combine  the  destructive  capabilities  of  a  conventional 
force  with  the  elusiveness  of  a  guerrilla."  [Ref.  13:p.  A-3]  With  additional  references  to 
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modernizing  our  capabilities"  and  the  need  to  "overcome  challenges  in  the  areas  of 
battlefield  mobility",  OMFTS  firmly  identifies  the  deficiencies  of  existing  amphibious 
vehicles,  and  establishes  the  justification  for  advanced  amphibian  assault  capabilities 
without  specifically  identifying  the  AAAV  by  name.  [Ref.  13:p.  A-5] 

The  evolution  of  amphibious  doctrine,  from  a  surface  assault  perspective, 
flourished  in  the  1930's,  solidified  in  the  1940's,  and  then  languished  for  nearly  40  years. 
Only  in  the  last  ten  years  has  amphibious  doctrine  been  revitalized  by  a  significant  shift 
in  the  operational  environment.  As  will  be  seen  next,  this  evolving  doctrine  drove  new 
equipment  requirements  for  its  support. 

C.  EVOLUTION  OF  ASSAULT  AMPHIBIAN  VEHICLES 

As  the  doctrine  for  amphibious  warfare  evolved  over  the  past  70  years,  so  did  the 
equipment  required  to  support  that  doctrine.  In  the  late  1 920’s  and  early  1930's,  the  Navy 
and  the  Marine  Corps  conducted  several  amphibious  exercises  in  Hawaii,  the  Caribbean, 
and  Quantico,  Virginia.  [Ref.  14]  The  results  of  these  exercises  concluded  that  existing 
landing  craft,  though  successful  in  transporting  troops  and  limited  equipment  from  ship  to 
shore,  lacked  the  capability  to  cross  coral  reefs  that  surrounded  many  Pacific  Islands. 
Additionally,  the  landing  craft  did  not  allow  Marines  to  push  equipment  and  supplies 
forward  off  the  exposed  beach  and  out  of  enemy  direct  fire  range.  [Ref.  3:pp.  15-16] 

Some  initial  tests  were  conducted  on  the  Christie  Amphibious  Tank  in  1924  and 
again  on  the  British  Vicker- Armstrong  Light  Amphibious  Tank  in  1932.  Both  designs 
were  rejected  because  of  their  questionable  seaworthiness,  and  their  slow  water  speeds. 
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[Ref.  15:pp.  17-33]  The  original  focus  on  these  new  vehicles  was  for  a  modified  tank  to 
become  amphibious  in  nature.  This  would  provide  the  first  wave  ashore  the  capability  to 
fill  a  fire  support  gap  between  naval  gunfire  and  close  air  support.  [Ref.  16:p.  8]  The 
Marine  Corps  soon  realized  that  it  must  coordinate  its  search  for  better  equipment,  and  in 
late  1933  the  Marine  Corps  Equipment  Board  was  established  in  Quantico,  Virginia.  The 

board  s  purpose  was  to  test  and  develop  material  for  landing  operations  and  expeditionary 
service.  [Ref.  14] 

In  1937,  a  Navy  Admiral  passed  to  the  Commandant  of  the  Marine  Corps  a  copy 
of  Life  Magazine  that  contained  an  article  titled,  "Roebling's  Alligator  for  Florida 
Rescue.  The  article  described  Donald  Roebling's  development  of  a  tracked  amphibious 
vehicle  designed  for,  "...transporting  victims  through  the  swamps,  over  drowned  roads, 
[and]  across  debris  filled  bayous."  [Ref.  15:p.  34]  The  Commandant  directed  the 
Equipment  Board  to  review  the  "Alligator"  for  potential  as  a  military  vehicle.  After  two 
years  of  review,  test,  and  evaluation,  Army,  Navy,  and  Marine  Corps  officials  were  so 
impressed  that  Food  and  Machinery  Corporation  (FMC)  was  contracted  to  build  a 
modified  Roebling  vehicle  for  full-scale  production.  The  first  Landing  Vehicle  Tracked 
(LVT(l))  came  off  the  assembly  line  in  July  1941,  and  the  1st  Amphibian  Tractor 
(AmTrac)  Battalion  of  the  1st  Marine  Division  was  fully  operational  by  16  February  1942. 
[Ref.  15:pp.  34-42] 

The  LVT(l)  was  strictly  a  cargo  variant  amphibian  armed  with  a  .50  and  .30 
caliber  machine  gun  for  self-protection.  The  Marine  Corps,  however,  continued  its 
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pursuit  of  an  armored  version  of  the  LVT  that  would  have  a  turret  with  a  37-mm  gun. 
[Ref.  16:p.  8]  The  LVT(l)  soon  found  itself  in  combat  in  WWII  during  operations  in 
Guadalcanal.  The  vehicle  proved  its  versatility  as  a  cargo  handler  by  shuttling  supplies 
from  ship  to  shore,  transporting  wounded  Marines,  providing  protective  fires,  towing 
artillery  into  position,  and  assisting  in  the  installation  of  a  bridge.  [Ref.  16:p.  9]  During 
this  period  in  the  United  States,  the  Marine  Corps  developed  an  improved  cargo  variant 
known  as  the  LVT(2),  or  "Water  Buffalo".  It  also  designed  and  produced  an  armored 
version  of  the  LVT(2)  designated  the  LVT(A)2,  and  a  turreted  version  of  the  LVT(A)2 
known  as  the  LVT(A)1.  [Ref.  16:p.  9] 

Despite  only  having  the  LVT(l)  and  the  LVT(2)  during  the  battle  for  Tarawa,  the 
2nd  Marine  Division  would  not  have  been  successful  without  out  them.  The  significant 
lesson  from  that  battle  was,  "The  need  for  more  LVTs  with  better  armor  and  an  armored 
fire  support  variant..."  [Ref.  16:p.  10]  The  Marine  Corps  was  convinced  of  the  value  of 
the  LVT  for  its  campaign  in  the  Central  Pacific,  and  was  sure  that,  "No  other  vehicle 
could  offer  protection  and  mobility  in  heavy  surf  with  full  loads  of  men  and  equipment 
and  rapidly  transition  from  water  to  land  operations."  [Ref.  16:p.  9]  Figure  1  shows  a 
LVT(2)  in  the  foreground  and  a  LVT  (1)  (Number  49)  following  the  battle  of  Tarawa. 
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Figure  1.  LVT(l)  and  LVT(2)  ashore  at  Tarawa.  [Ref.  38:p.  17] 


As  the  war  progressed,  the  lessons  from  combat  in  the  Marshall  Islands  indicated 
the  need  for  troop  carrying  LVTs  with  a  cargo  ramp  and  better  bilge  pumps  for 
discharging  water.  The  need  for  an  armored  variant  with  greater  firepower  than  the 

37mm  was  also  determined.  FMC  developed  and  produced  the  LVT(3)  and  the  LVT(4) 

* 

with  a  ramp  for  easier  loading  and  unloading  of  personnel,  equipment,  and  supplies. 
FMC  also  developed  and  produced  the  LVT(A)4  armed  with  a  75mm  howitzer  assault 
gun  and  later  the  LVT(A)5  with  a  gyro-stabilized  75mm  howitzer.  These  variants  were 
used  for  direct  fire  missions  upon  landing,  and  then  provided  indirect  fire  once  ashore. 
[Ref.  16:p.  11]  Figure  2  shows  LVT(A)4s  as  they  move  toward  shore  in  the  battle  of 


Okinawa. 


Figure  2.  LVT(A)4s  move  toward  shore  at  Okinawa  in  1945.  [Ref.  39:p.  14] 

By  the  war's  end  in  1945,  the  Marine  Corps  had  grown  from  the  Is1  Amphibious 

Tractor  Battalion  to,  "...three  armored  amphibian  battalions  equipped  with  the  LVT(A)4 

and  nine  cargo  amtrac  battalions  equipped  with  LVT(3)s  or  LVT(4)s."  [Ref.  17:p.  71] 

With  the  surrender  of  Japan,  "...LVT  production  that  over  the  preceding  4  years  had 

delivered  18,816  amphibian  tractors  of  all  models..."  came  to  an  end.  [Ref.  17:p.  71] 

In  1948,  1200  LVT(3)s  underwent  modification.  The  cargo  area  was  covered  by 

folding  metal  doors  to  shield  passengers  from  enemy  grenades,  and  from  breaking 

seawater.  Additionally,  a  machine  gun  cupola  was  mounted  top-center  and  forward  near 

the  bow.  [Ref.  15:p.  248]  The  resulting  LVT(3)C  variant  and  the  LVT(A)5  variant: 

...bore  the  brunt  of  fighting  in  Korea,  functioning  more  in  the  role  of  an 
armored  personnel  carrier  on  land  than  an  amphibious  vehicle  in  water¬ 
borne  landings  because  the  Korean  struggle  used  the  United  States  Marine 
Corps  as  much  for  its  infantry  power  as  for  its  amphibious  capability. 

Instead  of  short,  sharp  fights  for  islands,  the  Corps  operated  nearly 
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continuously  on  land  on  the  Korean  Peninsula,  requiring  the  LVT  to  assist 
more  overland  than  over  water.  [Ref.  15:p.  248] 

The  link  between  the  changing  operational  environment  of  the  LVT  in  Korea  and 
changing  requirements  for  a  new  LVT  was  established  with  the  production  of  the  LVT(5) 
family  of  vehicles  in  1953.  The  LVT(5)  was  a  significant  departure  in  design  from 
previous  WWII  variants  of  the  LVT.  Much  larger  in  size,  and  capable  of  carrying  30-34 
combat  loaded  Marines,  the  LVT(P)5  nearly  doubled  the  LVT(3)'s  personnel  capacity. 
[Ref.  16:p.  13] 

The  LVT(5)  family  of  vehicles  represented  the  second  major  generation  of  LVTs 
and  consisted  of  special  variants  each  with  its  own  mission.  The  LVT(P)5  was  a 
personnel  and  cargo  carrier.  The  LVT(C)5  was  a  command  and  control  vehicle  specially 
designed  as  a  mobile  command  post  during  amphibious  landings  and  operations  ashore. 
The  LVT(R)1  was  a  recovery  and  maintenance  vehicle  with  winches,  a  crane,  a  welding 
rig,  and  special  maintenance  equipment.  The  LVT(E)1  was  an  combat  engineer  variant 
designed  for  minefield  breaching.  This  vehicle  had  a  large  mine  plow  on  the  front,  and  a 
rocket-propelled  line  charge  inside.  When  deployed  and  detonated,  the  line  charge  would 
clear  a  350-foot  long  lane  through  a  minefield  by  sympathetically  detonate  the  buried 
mines.  Finally,  the  LVT(H)6  was  a  fire  support  vehicle  with  a  specially  designed  turret 
containing  a  105mm  howitzer.  [Ref.  15:pp.  248-254]  Figure  3  shows  a  LVT(R)1  with  its 
crane  in  the  extended  position. 
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Figure  3.  LVT(R)1  with  crane  in  extended  position. 


The  LVT(5)  family  of  vehicles  saw  extensive  action  in  Vietnam.  They  carried  out 
amphibious  landings,  fire  support  missions,  resupply  missions,  mechanized  patrolling, 
and  medical  evacuations.  Combat  use  of  the  LVT(5)  exposed  several  deficiencies 
including  flammable  gas  tanks  in  the  vehicle’s  bottom  deck,  fuel  and  hydraulic  leaks,  low 
water  speed  and  maneuverability,  and  failing  parts.  [Ref.  15:p.  255]  Combined  with  its 
20-year  age,  these  deficiencies  eventually  drove  the  Marine  Corps  to  seek  a  new  series  of 
LVTs.  [Ref.  16:  p.  14] 

The  LVT(7)  family  of  vehicles  constitutes  the  third  and  present  major  generation 
of  LVTs,  and  represents  yet  another  radical  departure  in  design.  The  vehicle  was 
designed  to  optimize  its  speed  in  the  water  through  the  use  of  water  jet  propulsion  units, 
and  through  an  angled  front-hull  shape  to  reduce  drag.  This  design  gave  the  vehicle 
greater  buoyancy  and  increased  maneuverability  in  the  water,  and  was  based  on  the 
original  operational  requirement  to  operate  80  percent  of  the  time  in  the  water  and  20 
percent  of  the  time  on  land.  [Ref.  16:p.l4]  Previous  LVTs  were  propelled  through  the 
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water  simply  by  having  the  specially  designed  tracks  continue  to  turn,  pushing  the  vehicle 
forward.  [Ref.  16:p.  14]  Table  1  shows  a  comparison  of  the  dimensions  and  speeds  of  the 
LVT(3),  LVT(P)5,  and  the  LVT(P)7.  The  LVT(7)  family  of  vehicles  is  much  smaller 
than  the  LVT(5)  family  and  only  consists  of  the  personnel  LVT(P)7,  command  and 
control  LVT(C)7,  and  recovery  LVT(R)7  variants. 


Characteristics 

LVT(3) 

wmmivMmm 

Length 

24'  6” 

29.  8m 

26' 

Width 

11'  2" 

11'  8.5" 

10'  3.75" 

Height  (top  of  vehicle) 

9'  11" 

8'  7.5" 

9'  9" 

Weight  (Empty) 

26,600  lbs. 

69,780  lbs. 

40,000  lbs. 

Cargo  Capacity 

12,000  lbs. 

12,000  lbs.  (water) 
18,000  lbs.  (land) 

10,000  lbs. 

Speed,  Water 

6  mph 

6.8  mph 

8.4  mph 

Speed,  Land 

17  mph 

30  mph 

40  mph 

Engine 

Cadillac  V-8  (2) 

Continental  V-12  (1) 

Detroit  Diesel 

each  220HP 

810  HP 

400  HP 

Table  1.  LVT(3),  LVT(P)5,  and  LVT(P)7  comparison.  [Ref.  15:pp.  254,256] 


The  Marine  Corps  increased  its  training  in  combined  arms  and  desert  warfare 
during  the  late  1970's  and  1980's  at  the  Marine  Corps  Air  Ground  Combat  Center, 
Twenty  Nine  Palms,  California.  These  Combined  Arms  Exercises  (CAX)  gradually 
shifted  the  role  of  the  LVT(7)  toward  that  of  an  armored  personnel  carrier.  The  exercises 
showed  that  the  Marines  needed  improvements  in  the  LVT(7)'s  survivability.  [Ref.  16:p. 
14]  The  shift  toward  mechanized  operations  changed  the  operational  ratios  of  the  LVT(7) 
to  20  percent  waterborne  operations  and  80  percent  land  operations.  [Ref.  19:  p.  2] 
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The  planned  service  life  of  the  LVT(7)  vehicle  was  ten  years.  However,  since  no 
replacement  existed,  the  Marine  Corps  began  a  Service  Life  Extension  Program  (SLEP) 
in  1983  to  ensure  operability  through  1994.  [Ref.  3:p.  20]  In  addition  to  modernizing  853 
LVT(P)7s,  77  LVT(C)7s,  and  54  LVT(R)7s,  the  Marine  Corps  acquired  333  of  the  newly 
designated  LVT7A1  vehicles.  [Ref.  17:p.  76]  The  SLEP  involved  converting  the  turret 
from  hydraulic  power  to  electrical  power,  and  replacing  the  Detroit  Diesel  engine  with  a 
newer  and  more  powerful  V-8  turbocharged  Cummins  Diesel  engine.  Other 
modifications  involved  changes  to  the  fuel  tanks,  suspension,  and  communications.  Soon 
after  the  completion  of  the  SLEP,  the  vehicle  was  redesignated  the  AAV7A1  family. 
[Ref.  16:p.  14] 

The  December  1991  issue  of  the  Marine  Corps  Gazette  describes  the  next  series  of 

vehicle  modifications  and  improvements  as  follows: 

Beginning  in  1987,  a  series  of  product  improvements  and  a  rotating 
inspection  cycle  were  initiated  and  have  pushed  the  life  expectancy  well 
into  the  next  decade  when  a  replacement  system  is  expected  to  be  fielded. 

The  product  improvement  program  (PIP)  combines  a  number  of  initiatives 
to  enhance  vehicle  mobility,  firepower,  and  survivability.  These 
initiatives,  broken  into  three  separate  blocks,  are  being  phased  in  gradually 
over  a  number  of  years.  [Ref.  18:p.  4] 

The  Block  1  upgrades  consist  of  [Ref.  18:p.  4]: 

•  Replacing  the  electric  drive  weapons  station  and  its  M-85  .50  Cal  machine 
gun  with  an  Upgunned  Weapons  Station  with  a  Mk-19  40mm  machine  gun 
and  a  M2HB  .50  Cal  heavy  machine  gun. 

•  Adding  P-900  applique  armor  to  the  sides  of  the  AAV  to  give  the  vehicle 
protection  from  incoming  rounds  by  interrupting  and  diverting  their  flight 
path. 
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•  Adding  a  rotary  bow  plane  kit  to  the  front  of  the  vehicle  to  aid  in  waterborne 
mobility  by  enhancing  trim. 

The  Block  2  upgrades  consist  of  [Ref.  18:p.  4]: 

•  Adding  an  automatic  fire-sensing  and  suppression  system  capable  of 
extinguishing  flames  within  microseconds. 

•  Replacing  the  P-900  armor  with  Enhanced  Armor  Applique  Kits  (EAAK). 
EAAK  covers  more  surface  area  and  provides  a  greater  protection  from 
incoming  rounds  than  does  the  P-900. 

•  Minor  modifications  to  the  suspension  and  upgrades  to  the  transmission. 

The  Block  3  upgrades  described  in  the  Marine  Corps  Gazette  never  materialized 

as  envisioned  but  evolved  into  the  Reliability,  Availability  Maintainability/Rebuild  to 
Standard  (RAM/RS)  program.  The  addition  of  the  turret,  armor,  and  bow  plane  resulted 
in  an  increase  of  weight  for  the  AAVP7A1  from  40,000  pounds  to  61,300  pounds  fully 
loaded.  [Ref.  20  :p.  1]  The  detailed  test  plan  for  the  AAV  RAM/RS  states,  “The 
increased  weight  has  resulted  in  a  reduced  horsepower  to  weight  ratio,  increased 
maintenance  costs  on  the  suspension,  limited  supply  support,  and  decreased 
performance.”  [Ref.  20:p.  1]  This  degradation  of  performance  made  the  AAV7A1 
unable  to  perform  to  its  original  specifications.  Additionally,  Marine  Corps  attempts  to 
maintain  the  AAV’s  readiness  through  the  Inspect  and  Repair  Only  as  Necessary 
(IROAN)  program  became  prohibitively  costly.  [Ref.  20:p.  1]  Figure  4  shows  an 
AAVP7A1  with  Block  1  and  Block  2  upgrades  including  Upgunned  Weapon  Station, 
Enhanced  Applique  Armor  Kit  (EAAK),  and  bow  plane. 
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Figure  4.  AAVP7A1  with  Upgunned  Weapons  Station,  EAAK,  and  Bow  Plane. 


The  AAV7A1  RAM/RS  will  undergo  significant  modifications  to  the  hull  in  order  to 
accommodate  a  new  suspension  system  and  new  engine,  both  derivatives  of  the  U.  S.  Army’s 
M2  Bradley  Fighting  Vehicle.  The  new  suspension  will  raise  the  present  ground  clearance  of  12 
inches  back  to  the  original  16  inches.  A  Cummins  Model  VT903-525  horsepower  engine  will 
replace  the  current  400  horsepower  engine,  thus  regaining  the  17  to  1  horsepower  to  ton  ratio. 
[Ref.  19:p.  3]  The  detailed  test  plan  further  states, 


The  AAV7A1  RAM/RS  will  enable  the  vehicle  to  operate  at  or  near  its 
original  performance  specifications  and  allow  it  to  better  perform  its 
assigned  missions  indicated  in  the  original  Required  Operational 
Capability  (ROC)  document.  These  improvements  will  provide  a  more 
reliable  and  less  costly  platform  which  will  allow  the  AAV7A1  to 
continue  as  the  primary  tactical  ground  mobility  asset  for  the  Marine 
Corps  until  the  introduction  of  the  new  Advanced  Amphibious  Assault 
Vehicle.  (AAAV).  [Ref.  20:p.  1] 

The  mission  of  the  AAV7A1  family  of  vehicles  remains,  “...to  land  the  surface 
elements  of  the  landing  force  and  their  equipment  in  a  single  lift  from  assault  shipping 


during  amphibious  operations  to  inland  objectives  and  to  conduct  mechanized  operations 
and  related  combat  support  in  subsequent  operations  ashore.”  [Ref.  19:p.  1]  However, 
even  with  the  RAM/RS  upgrades,  the  AAV7A1  family  of  vehicles  remains  deficient  in 
meeting  the  needs  of  the  Marine  Corps  as  it  moves  toward  implementing  the  Operational 
Maneuver  From  The  Sea  (OMFTS)  doctrine. 

The  Operational  Requirements  Document  (ORD)  for  the  Advanced  Amphibian 
Assault  Vehicle  (AAAV)  (No.  MOB  22.1)  specifically  states,  "The  deficiencies  and 
design  constraints  of  the  AAV7A1  render  it  unsuitable  for  the  execution  of '...From  the 
Sea'  and  OMFTS  amphibious  operations  in  the  year  2005  and  beyond."  [Ref.  2:p.  9]  A 
draft  copy  of  the  System  Concept  Paper  (SCP)  for  the  Advanced  Amphibious  Assault 
(AAA)  program  indicates  that  these  deficiencies  were  tentatively  identified  in  the 
Mission  Area  Analysis  of  Mission  Area  232.3,  Ship  to  Shore  Movement  (Secret 
NOFORN)  dated  24  August  1987.  [Ref.  21  :Encl.  1]  The  Draft  SCP  goes  on  to  state  that, 
"Significant  shortfalls  in  land  and  sea  mobility,  speed,  and  agility  along  with  serious 
deficiencies  in  offensive  firepower  (lethality)  and  inherent  armor  protection  levels 
(survivability),  have  been  confirmed  through  force-on-force  analysis  using  DIA  [Defense 
Intelligence  Agency]  validated  threat  projections."  [Ref.  21:Encl.  1] 

The  original  AAAV  ORD  identifies  1 1  major  deficiencies  with  respect  to  the 
operational  capabilities  and  survivability  of  the  AAV7A1.  They  are  as  follows:  [Ref. 

2:pp.  10-11] 

•  Water  Mobility.  The  AAV7A1  possesses  inadequate  water  speed  to  support 
the  "From  the  Sea"  and  OMFTS  concepts. 
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Firepower 


The  AAVP7A1  does  not  have  a  weapon  system  with  sufficient  lethality  to 
defeat  light  armored  combat  vehicles  (BTR,  BRDM,  BMP)  projected  for  the  2005-2025 
time  frame. 


-  The  AAVP7A1  does  not  have  a  system  that  enables  the  gunner/vehicle 
commander  to  sight  his  weapons  system  during  reduced  light  conditions  or  through 
battlefield  obscurants  (e.g.,  smoke)  to  the  effective  range  of  weapons. 

-  The  weapon  systems  in  the  AAVP7A1  are  not  stabilized  to  allow  accurate 
employment  of  the  weapons  against  stationary  and  moving  enemy  targets  while  the 
vehicle  is  moving  on  land  and  in  the  water. 

•  Armor  Protection 


-  The  AAV7A1  has  minimal  integral  hull  side  armor  protection  against 
7.62mm  Armor  Piercing  (AP)  munitions  fired  at  greater  than  300  meters  (1000  feet)  and 
overhead  protection  from  105mm  artillery  bursts  15  meters  (50  feet)  from  the  vehicle. 

The  AAV7A1  can  only  achieve  14.5mm  armor  protection  on  the  sides  at 
300  meters  and  overhead  protection  against,  for  the  troop  compartment,  152mm  artillery 
fragments  at  15  meters  from  the  vehicle  with  the  application  of  bolt-on  armor  which  adds 
2040  kg  (4500  lbs)  to  overall  vehicle  weight  and  is  time  consuming  to  install. 
Additionally,  the  added  weight  reduces  mobility,  produces  a  negative  effect  on  the 
suspension,  and  adversely  effects  the  power  train  and  suspension  life. 

-  The  AAV7A1  possesses  inadequate  armor  protection  against  chemical 
energy  weapons  (e.g.,  RPG). 

•  Land  Mobility.  The  AAV7A1  lacks  the  mobility  to  operate  with  the  M-1A1 
main  battle  tank. 

•  Signature.  The  AAV7A1  family  of  vehicles  possess  significant  adverse 
visual,  aural,  magnetic,  infrared,  and  electromagnetic  signatures. 

•  Fire  Prevention  /  Suppression.  The  AAV7A1  does  not  have  a  multi-incident 
fire  suppressive  capability  that  protects  the  crew  and  embarked  personnel  against 
catastrophic  fires  resulting  from  ignition  of  the  fuel  or  fuel  cell  by  munitions  or 
incendiary  rounds  penetrating  the  vehicle  hull. 
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•  Directed  Energy.  The  AAV7A1  lacks  directed  energy  countermeasures  to 
protect  against  directed  energy  weapons. 

•  NBC  Protection 

-  The  AAVP7A1  does  not  possess  a  collective  protection  system  that  will 
allow  the  crew  and  embarked  personnel  to  survive  in  NBC  [Nuclear,  Biological, 
Chemical]  environments. 

The  AAV7A1  does  not  have  a  NBC  detection  or  monitoring  capability. 

-  The  present  level  of  nuclear  hardening  protection  in  the  AAV7Al's 
electrical  and  electronic  equipment  will  not  protect  equipment  from  the  current  threat 
capability. 


•  Night  Driving.  The  driver's  station  of  the  AAV7A1  does  not  provide  the 
driver  with  the  capability  to  see  through  obscurants  or  operate  at  high  speeds  in  adverse 
weather  or  at  night. 

•  Anti-Tank  Guided  Missile  (ATGM)  Countermeasure.  The  AAV7A1  does  not 
possess  a  countermeasure  capability  to  defeat  ATGM  guided  systems. 

•  Current  Operational  Deficiencies  of  the  AAVC7A1 

-  Accurate  and  continuous  location  of  friendly  forces  is  not  available. 

-  The  vehicle  has  inadequate  ventilation  for  heat  generated  by 
communications  equipment. 

-  Communications  equipment  fails  due  to  exposure  to  the  amphibious  (salt¬ 
water)  environment. 

The  current  AAVC7A1  does  not  meet  all  the  net  requirements  of  the 

supported  unit. 

The  above  listed  deficiencies  in  the  AAV7A1  family  of  vehicles  establish  the 
requisite  capabilities  of  the  next  generation  of  amphibious  vehicles,  the  AAAV.  Further 
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more,  the  requisite  capabilities  will  establish  a  benchmark  from  which  the  new  vehicle 
will  be  tested. 

D.  CHAPTER  SUMMARY 

This  chapter  examined  the  evolution  of  amphibious  doctrine  and  the  evolution  of 
the  amphibious  vehicles  that  met  the  needs  of  that  doctrine.  The  chapter  serves  as  a 
foundation  for  understanding  the  Marine  Corps’  stated  requirement  for  the  AAAV.  The 
following  chapter  describes  the  DoD’s  acquisition  process. 
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III.  DEPARTMENT  OF  DEFENSE  ACQUISITION  OVERVIEW 

A.  INTRODUCTION 

This  chapter  provides  the  reader  with  an  overview  of  the  Department  of  Defenses’ 
(DoD’s)  acquisition  system,  and  recent  acquisition  reform  efforts.  It  will  cover  the  DoD 
5000  Series  of  directives  and  instructions,  key  individuals  and  forums  in  the  acquisition 
process,  the  requirements  generation  process,  and  the  acquisition  management  process. 
The  chapter  will  address  these  concepts  from  the  major  defense  acquisition  program 
(MDAP)  level,  specifically  for  Acquisition  Category  1(D)  (ACAT  1(D))  programs,  as  this 
is  the  category  for  the  AAAV. 

B.  DOD  ACQUISITION  SYSTEM  AND  ACQUISITION  REFORM 

The  defense  acquisition  system,  “is  a  single  uniform  system  whereby  all 
equipment,  facilities,  and  services  are  planned,  developed,  acquired,  maintained,  and 
disposed  of  by  the  Department  of  Defense  (DoD)."  [Ref.  22:p.  1]  The  system  provides 
the  DoD  a  standardized  framework  for  the  acquisition  of  weapon  systems  that  allow  the 
DoD  to  meet  and  conduct  its  mission.  This  system  is  greatly  influenced  by  the  Executive 
Branch  that  sets  policy,  the  Legislative  Branch  that  controls  funding  and  legislation,  and 
the  Defense  Industry  that  ultimately  produces  weapon  systems.  Often  these  three 
elements  create  an  environment  of  significant,  diverse  and  competing  interests.  [Ref. 
22:pp.  1-5]  The  ultimate  goal  of  the  defense  acquisition  system  is  to,  “...acquire  quality 
products  that  satisfy  the  needs  of  the  operational  user  with  measurable  improvements  to 
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mission  accomplishment,  in  a  timely  manner,  at  a  fair  and  reasonable  price.”  [Ref.  23  :p. 

3] 

Defense  systems  acquisition  get  its  authority  from  a  multitude  of  statutes, 
executive  directions,  Office  of  Management  and  Budget  (OMB)  Circular  A- 109,  and  the 
Federal  Acquisition  Regulation  (FAR).  Congressional  dissatisfaction  with  DoD 
acquisition  procedures  has  resulted  in  a  number  of  initiatives  and  legislation  to  reform  the 
overall  acquisition  process.  Most  significant  of  these  were  the  1991  Department  of 
Defense  Authorization  Act,  the  1994  Federal  Acquisition  Streamlining  Act  (FAS A),  and 
the  1996  National  Defense  Authorization  Act.  The  basic  intent  of  these  acts  was  to 
simplify  the  procurement  procedures  and  to  restructure  the  DoD  acquisition  organization 
and  workforce.  [Ref.  22:pp.  1 1-13] 

C.  DOD  5000  SERIES 

One  of  the  measures  taken  to  simplify  the  procurement  procedures  was  the 
consolidation  of  numerous  DoD  acquisition  directives  and  instructions  into  a  new  DoD 
5000  Series.  The  provisions  of  OMB  A- 109  were  implemented  in  March  of  1996  in  two 
primary  documents,  DoD  Directive  5000.1,  Defense  Acquisition  and  DoD  Regulation 
5000.2-R,  Mandatory  Procedures  for  Major  Defense  Acquisition  Programs  (MDAPs)  and 
Major  Automated  Information  System  (MAIS)  Acquisition  Programs.  [Ref.  22 :p.  15] 

DoD  5000.1  identifies  three  broad  policy  principles  applicable  to  all  defense 
acquisition  programs,  three  primary  decision  support  processes,  and  key  acquisition 
individuals.  The  three  principles  are:  [Ref.  23:pp.  4-7] 
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•  Translating  operational  needs  into  stable,  affordable  programs. 

•  Acquiring  Quality  Products. 

•  Organizing  for  Efficiency  and  Effectiveness. 

One  of  the  elements  of  the  first  principle  is  to  provide  for  an  integrated  management 
framework.  The  three  primary  decision  support  processes  form  this  management 
framework.  Those  three  processes  are:  [Ref.  23  :p.  4] 

•  Requirements  Generation  Process. 

•  Acquisition  Management  Process. 

•  Planning,  Programming,  and  Budgeting  Process. 

As  can  be  seen  in  Figure  5,  the  three  processes  are  overlapping  and  related. 


Following  an  introduction  to  key  individuals  and  key  acquisition  forums,  the 
requirements  generation  process  and  acquisition  management  process  will  be  covered. 
Test  and  evaluation  generally  falls  into  the  overlap  area  of  these  two  processes. 


29 


D.  KEY  ACQUISITION  INDIVIDUALS  AND  FORUMS 

DoD  5000.1  describes  a  key  official  as,  “...a  DoD  official  who  is:  a  member  of 
the  streamlined  acquisition  chain  of  command  or  a  member  of  the  Defense  Acquisition 
Board.”  [Ref.  23]  A  forum  is  a  group  of  key  officials  responsible  for  acquisition  related 
decisions.  This  section  briefly  describes  the  responsibilities  of  key  acquisition  officials 
and  forums  pertinent  to  a  Marine  Corps  MDAP-level  program. 

1.  Deputy  Secretary  of  Defense 

The  Deputy  Secretary  of  Defense  approves  funding  for  new  acquisition  programs 
and  provides  general  affordability  planning  guidance  for  use  in  structuring  these 
programs.  The  Deputy  is  also  responsible  for  leading  the  Defense  Resources  Board 
(DRB).  [Ref.  23  :p.  9] 

2.  Under  Secretary  of  Defense  for  Acquisition  and  Technology  (USD(A&T)) 

The  Under  Secretary  of  Defense  for  Acquisition  and  Technology  is  the 
Department  of  Defense’s  Acquisition  Executive  for  Major  Defense  Acquisition  Programs 
(MDAPs).  The  USD(A&T)  supervises  the  entire  DoD  acquisition  system,  develops 
acquisition  program  guidance  and  ensures  compliance  with  established  policy  and 
procedures,  establishes  policies  for  the  training  and  development  of  acquisition  personnel, 
and  chairs  the  Defense  Acquisition  Board  (DAB).  [Ref.  22:p.  27]  Figure  6  presents  the 
DoD  Acquisition  Authority  Chain  as  it  pertains  to  MDAPs.  An  element  of  acquisition 
reform  was  to  consolidate  this  chain  and  provide  for  only  four  levels  of  authority.  [Ref. 
22:p.  23] 
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Figure  6.  DoD  Acquisition  Authority  Chain.  [Ref.  22:p.  24] 

3.  Secretary  of  the  Navy 

The  Secretary  of  each  military  department  is  responsible  for  ensuring  that  policies 
and  procedures  governing  the  operation  of  the  department's  acquisition,  requirements,  and 
budgeting  systems  are  effectively  implemented.  The  Secretary  will  designate  a  single, 
full-time  Acquisition  Executive  at  the  Assistant  Secretary  level  known  as  the  Component 
Acquisition  Executive  (CAE).  The  Secretary  will  also  select  Program  Executive  Officers 
(PEOs),  establish  a  centralized  system  for  selecting  Program  Managers  (PMs),  and 
charter  a  Component-level  systqm  of  acquisition  oversight  and  review.  [Ref.  23  :p.  9]  The 
Secretary  of  the  Navy  oversees  these  responsibilities  for  the  Marine  Corps. 


31 


4.  The  Vice  Chairman  of  the  Joint  Chiefs  of  Staff  (VCJCS) 

The  VCJCS  chairs  the  Joint  Requirements  Oversight  Council  (JROC),  vice-chairs 
the  Defense  Acquisition  Board,  and  represents  the  Commanders-in-Chiefs  of  the  Unified 
Combatant  Commands  on  acquisition  and  requirements  matters.  [Ref.  23 :p.  9] 

5.  Component  Acquisition  Executives  (CAEs) 

The  CAE  supervises  the  operation  of  the  acquisition  system  within  their 
respective  Component  and  are  responsible  for  enforcing  policies  established  by  the 
USD(A&T).  CAEs  also  serve  as  decision  authorities  for  assigned  programs.  [Ref.  23  :p. 
10] 

6.  Assistant  Secretary  of  the  Navy  for  Research,  Development,  and 
Acquisition  (ASN(RD&A)) 

The  ASN(RD&A)  is  the  senior  official  in  the  Department  of  the  Navy  responsible 
for  acquisition  matters.  Thus,  the  ASN(RD&A)  is  the  Navy  and  Marine  Corps’  CAE. 
[Ref.  22  :p.  25] 

7.  Program  Manager  (PM) 

PMs  manage  their  assigned  programs  in  a  manner  consistent  with  the  policies  and 
principles  articulated  in  DoD  5000.1,  DoD  5000.2R,  and  the  PM  Bill  of  Rights.  PMs 
provide  assessments  of  their  program’s  status  and  risk  to  higher  authorities  and  to  the 
user  or  user's  representative.  PMs  actively  manage  their  programs  within  approved 
resources,  program  cost,  performance,  and  schedule  parameters.  They  also  provide 
assessments  of  contractor  performance  to  higher  authorities.  [Ref.  23  :p.  10] 
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8.  Overarching  Integrated  Product  Team  (OIPT)  Leaders 

The  OIPT  provides  strategic  guidance  to  the  program  office,  resolve  issues,  and 
provide  an  independent  assessment  to  the  USD(A&T)  and  the  DAB  at  major  decision 
points,  using  information  gathered  through  the  Integrated  Product  Team  (IPT)  process. 
[Ref.  23 :p.  10]  “OIPTs  for  ACAT  ID  programs  are  led  by  the  appropriate  OSD  official. 
Typically  the  Director  of  Strategic  and  Tactical  Systems,  the  Assistant  Deputy  Under 
Secretary  of  Defense  (Space  and  Acquisition  Management),  or  the  Deputy  Assistant 
Secretary  of  Defense  (C3I  Acquisition).”  [Ref.  22:p.  33] 

9.  Defense  Resources  Board  (DRB) 

The  DRB  is  the  senior  DoD  resource  allocation  board  chaired  by  the  Deputy 
Secretary  of  Defense.  The  DRB  advises  the  Deputy  Secretary  on  major  resource 
allocation  decisions,  and  plays  a  significant  role  in  the  Planning,  Programming,  and 
Budgeting  System  process.  [Ref.  22:p.  29] 

10.  Defense  Acquisition  Board  (DAB) 

The  DAB  is  the  senior  DoD  acquisition  review  board  chaired  by  the  USD(A&T). 
It  advises  the  USD(A&T)  on  major  decisions  on  individual  acquisition  programs, 
acquisition  policies,  and  acquisition  procedures.  [Ref.  23  :p.  10]  The  USD(A&T)  will 
make  a  go  or  no-go  decision  about  the  continuation  of  a  system  based  on  cost,  schedule, 
and  performance  parameters  following  a  DAB  review.  The  USD(A&T)’s  decision  will 
be  published  as  an  Acquisition  Decision  Memorandum  (ADM).  [Ref.  22:p.  35] 


11.  Joint  Requirements  Oversight  Council  (JROC) 

The  JROC  is  chaired  by  the  VCJCS.  It  conducts  requirements  analyses,  validates 
mission  needs  and  key  performance  parameters,  develops  recommended  joint  priorities 
for  those  needs,  and  advises  the  Chairman  of  the  Joint  Chiefs  of  Staff  (CJCS)  on 
requirements  issues.  [Ref.  23  :p.  11]  The  JROC  will  also  review  MDAPs  at  each  of  the 
Milestones  prior  to  the  DAB  review.  [Ref.  22:p.  30] 

12.  Integrated  Product  Teams  (IPT) 

An  IPT  is  composed  of  representatives  from  all  appropriate  functional  disciplines 
working  together  to  build  successful  and  balanced  programs,  identify  and  resolve  issues, 
and  make  sound  and  timely  recommendations  to  facilitate  decision-making.  [Ref.  23  :p. 
11] 

The  key  individuals  and  forums  in  the  DoD  acquisition  system  ensure  that  the 
right  mission  need  is  validated,  the  right  material  solution  is  identified,  and  the  right 
system  is  produced  for  the  user.  The  requirements  generation  process  and  the  acquisition 
management  process  are  tools  that  the  key  individuals  use  to  accomplish  their  tasks. 

E.  REQUIREMENTS  GENERATION  PROCESS 

The  requirements  generation  process  is  a  method  used  by  each  service  in  the  DoD 
to  assess  the  capabilities  of  their  current  force  structure  and  its  ability  to  meet  the 
projected  threat.  The  services  do  this  assessment  in  light  of  opportunities  for 

i 

technological  advancement,  cost  savings,  and  changes  in  the  national  policy  or  the 
doctrine  used  to  support  that  policy.  [Ref.  22 :p.  39]  This  is  the  environment-doctrine- 
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technology  cycle  mentioned  at  the  beginning  of  the  chapter.  Each  service  conducts  this 
assessment  for  the  broad  category  of  warfighting  responsibility,  or  mission  area,  that  it 
has  cognizance  over.  The  Marine  Corps  Combat  Development  Command  (MCCDC) 
conducts  these  Mission  Area  Assessments  (MAA)  for  the  Marine  Corps.  [Ref.  22:p.  39] 
As  MCCDC  conducts  a  MAA,  it  may  determine  that  a  warfighting  deficiency 
exists,  or  an  opportunity  to  provide  new  capabilities  exists.  These  deficiencies  or 
opportunities  can  have  either  a  material  or  nonmaterial  solution.  Nonmaterial  solutions 
such  as  changing  or  improving  tactics,  doctrine,  or  training  are  preferred  because  of  lower 
costs  in  relation  to  a  material  solution.  If  a  deficiency  can  not  be  meet  by  a  nonmaterial 
solution,  then  the  requirement  for  a  material  solution  is  documented  in  a  Mission  Need 
Statement  (MNS).  The  MNS  describes  the  material  solution  in  broad  terms  and  specifies 
a  needed  capability,  not  a  specific  weapons  system.  [Ref.  22:p.  40] 

Once  a  MNS  for  a  major  defense  acquisition  program  (MDAP)  is  generated  by 
MCCDC  and  approved  by  the  Commandant  of  the  Marine  Corps,  it  gets  forwarded  to  the 
Joint  Requirements  Oversight  Council  (JROC)  for  validation  and  approval.  Validation  of 
the  MNS  by  the  JROC  confirms  that  the  need  to  correct  the  deficiency  or  exploit  the  new 
capability  exists  and  cannot  be  resolved  by  a  nonmaterial  solution.  Approval  simply 
means  that  the  JROC  concurs  that  the  need  is  valid  and  that  the  validation  process  is 
complete.  An  approved  MNS  for  a  MDAP  is  forwarded  by  the  JROC  to  the  Under 
Secretary  of  Defense  (Acquisition  and  Technology)  (USD(A&T)).  A  disapproved  MNS 
is  returned  to  the  originator.  [Ref.  22:p.  41]  If  the  USD(A&T)  concurs  with  the  MNS, 
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then  approval  to  enter  Milestone  0  is  granted.  “A  favorable  Milestone  0  decision  marks 
the  transition  from  the  requirements  generation  process  to  the  acquisition  process.”  [Ref. 
22 :p.  42] 

F.  ACQUISITION  MANAGEMENT  PROCESS 

The  acquisition  management  process  is  a  generic  model  that  all  MDAPs  follow  as 

they  progress  throughout  their  life  cycle.  The  model  consists  of  milestones  and  phases 

that  allow  for  review  of  the  program  by  key  individuals  so  they  can  monitor  and 

administrate  the  program’s  progress,  identify  problems  with  the  program,  and  make  any 

necessary  corrections  to  the  program.  [Ref.  22:pp.  45-46] 

DoD  5000.2-R  defines  a  major  milestone  as,  “...the  decision  point  that  separates 

the  phases  of  an  acquisition  program.”  [Ref.  5:p.  3]  For  a  MDAP  that  is  designated 

Acquisition  Category  1(D)  (ACAT  1(D)),  the  USD(A&T)  is  the  individual,  or  Milestone 

Decision  Authority  (MDA),  who  authorizes  the  program’s  entry  into  the  next  phase. 

The  DoD  5000.2-R  also  describes  the  acquisition  phase: 

All  the  tasks  and  activities  needed  to  bring  the  program  to  the  next  major 
milestone  occur  during  an  acquisition  phase.  Phases  provide  a  logical 
means  of  progressively  translating  broadly  stated  mission  needs  into  well- 
defined  system-specific  requirements  and  ultimately  into  operationally 
effective,  suitable,  and  survivable  systems.  [Ref.  5:p.  2] 

Figure  7  depicts  the  acquisition  management  process  model  with  its  various  phases  and 

milestones,  each  of  which  will  be  described  briefly. 
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Figure  7.  DoD  Acquisition  Management  Process  Model  [Ref.  22:p.  45] 


1.  Phase  0:  Concept  Exploration 

The  Concept  Exploration  (CE)  phase  consists  of  competitive  and  parallel  short¬ 
term  concept  studies.  DoD  5000.2R  states: 

The  focus  of  these  efforts  is  to  define  and  evaluate  the  feasibility  of 
alternative  concepts  and  to  provide  a  basis  for  assessing  the  relative  merits 
(i.e.  advantages  and  disadvantages,  degree  of  risk)  of  these  concepts  at  the 
next  milestone  decision  point.  Analysis  of  alternatives  shall  be  used  as 
appropriate  to  facilitate  comparisons  of  alternative  concepts.  The  most 
promising  system  concepts  shall  be  defined  in  terms  of  initial,  broad 
objectives  for  cost,  schedule,  performance,  software  requirements, 
opportunities  for  tradeoffs,  overall  acquisition  strategy,  and  test  and 
evaluation  strategy.  [Ref.  5:Part  1.4.2] 


2.  Phase  I:  Program  Definition  and  Risk  Reduction 

The  Program  Definition  and  Risk  Reduction  (PDRR)  phase  is  generally 

characterized  by  developing  measures  to  aid  in  reducing  the  risk  of  incorporating  new 

technologies  into  the  system.  [Ref.  22:p.  48]  DoD  5000.2R  further  states: 

During  this  phase,  the  program  shall  become  defined  as  one  or  more 
concepts,  design  approaches,  and/or  parallel  technologies  are  pursued  as 
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warranted.  Assessments  of  the  advantages  and  disadvantages  of 
alternative  concepts  shall  be  refined.  Prototyping,  demonstrations,  and 
early  operational  assessments  shall  be  considered  and  included  as 
necessary  to  reduce  risk  so  that  technology,  manufacturing,  and  support 
risks  are  well  in  hand  before  the  next  decision  point.  Cost  drivers,  life- 
cycle  cost  estimates,  cost-performance  trades,  interoperability,  and 
acquisition  strategy  alternatives  shall  be  considered  to  include 
evolutionary  and  incremental  software  development.  [Ref.  5:Part  1.4.3] 

3.  Phase  II:  Engineering  and  Manufacturing  Development 

“The  primary  objectives  of  this  phase  are  to:  translate  the  most  promising  design 
approach  into  a  stable,  interoperable,  producible,  supportable,  and  cost-effective  design; 
validate  the  manufacturing  or  production  process;  and,  demonstrate  system  capabilities 
through  testing.”  [Ref.  5:Part  1.4.4]  The  testing  conducted  during  Engineering  and 
Manufacturing  Development  (EMD)  is  designed  to  ensure  that  specifications  are  met  and 
to  ensure  the  system  is  operationally  effective  and  suitable.  [Ref.  22 :p.  49]  Additionally, 
Low  Rate  Initial  Production  (LRIP)  will  occur  while  the  EMD  phase  is  continuing  as 
both  the  test  results  and  design  fixes  or  upgrades  are  incorporated  into  the  system.  [Ref. 
5:Part  1.4.4] 


4.  Phase  III:  Production,  Fielding/Deployment,  and  Operational  Support 

The  Production,  Fielding/Deployment,  and  Operational  Support  (PF/DOS)  phase 
will  often  overlap  with  the  EMD  phase,  especially  when  Low  Rate  Initial  Production 
(LRIP)  is  an  element  of  the  program’s  acquisition  strategy.  [Ref.  22:p.49]  DoD  5000.2R 
states  that  the  objectives  of  the  PF/DOS  phase  are  to,  “achieve  an  operational  capability 
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that  satisfies  mission  needs.”  By  satisfying  a  mission  need,  the  acquisition  management 
process  completes  the  cycle  that  began  in  the  requirements  generation  process. 

DoD  5000.2R  further  states  that: 

Deficiencies  encountered  in  Developmental  Test  and  Evaluation  (DT&E) 
and  Initial  Operational  Test  and  Evaluation  (IOT&E)  shall  be  resolved  and 
fixes  verified  in  Follow-on  Operational  Test  and  Evaluation  (FOT&E). 

During  fielding  /  deployment  and  throughout  operational  support,  the 
potential  for  modifications  to  the  fielded  /  deployed  system  continues. 

[Ref.  5  Tart  1.4.5] 

Upon  completion  of  a  system’s  useful  life,  the  Program  Manager  (PM)  must 
ensure  it  is  demilitarized  and  disposed.  The  demilitarization  and  disposal  process  must 
be  controlled  and  meet  environmental,  safety,  and  security  standards.  Disposal  of  the 
system  completes  the  acquisition  cycle.  [Ref.  22:p.49]  As  seen,  the  three  acquisition 
phases  allow  for  the  completion  of  the  tasks  and  activities  needed  to  bring  the  program  to 
the  next  major  milestone. 

5.  Milestone  0:  Approval  to  Conduct  Concept  Studies 

The  USD(A&T)  will  convene  a  Milestone  0  Defense  Acquisition  Board  (DAB)  to 
review  the  mission  needs  statements  (MNS)  validated  by  the  JROC.  The  USD(A&T)  will 
also  identify  possible  materiel  alternatives  to  be  examined,  authorize  concept  studies, 
designate  the  lead  organization,  and  establish  exit  criteria  for  Phase  0.  [Ref  22  :p.  47]  A 
favorable  Milestone  0  decision  does  not  mean  that  a  new  acquisition  program  has  been 
initiated,  it  simply  means  that  concepts  for  material  solutions  to  a  mission  need  can  be 
explored.  [Ref.  5:Part  1.5.1] 
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6.  Milestone  I:  Approval  to  Begin  a  New  Acquisition  Program 

“The  purpose  of  the  Milestone  I  decision  point  is  to  determine  if  the  results  of 
Phase  0  warrant  establishing  a  new  acquisition  program  and  to  approve  entry  into  Phase  I, 
Program  Definition  and  Risk  Reduction.”  [Ref.  5:Part  1 .5.2]  At  this  milestone,  the 
PM  shall  submit  an  acquisition  strategy,  an  Acquisition  Program  Baseline  (APB),  and 
proposed  Phase  I  exit  criteria  to  the  MDA  for  approval. 

From  a  test  and  evaluation  perspective,  the  Director  of  Operational  Test  and 
Evaluation  (DOT&E)  and  the  Director  of  Test,  Systems  Engineering  and  Evaluation 
(DTSE&E)  shall  approve  the  PMO’s  Test  and  Evaluation  Master  Plan  (TEMP)  for  all 
Office  of  the  Secretary  of  Defense  (OSD)  test  and  evaluation  oversight  programs.  Should 
full-up,  system-level  Live  Fire  Test  and  Evaluation  (LFT&E)  be  deemed  unreasonably 
expensive  and  impractical,  the  PM  shall  request  a  waiver  and  submit  alternative  LFT&E 
plans  for  approval  by  Milestone  II.  [Ref.  5:Part  1.5.2] 

7.  Milestone  II:  Approval  to  Enter  Engineering  and  Manufacturing 
Development 

“The  purpose  of  the  Milestone  II  decision  point  is  to  determine  if  the  results  of 
Phase  I  warrant  continuation  of  the  program  and  to  approve  entry  into  Engineering  and 
Manufacturing  Development  (applies  to  both  hardware  and  software).”  [Ref.  5:Part 
1.5.3]  At  this  milestone,  the  PM  will  submit  an  updated  acquisition  strategy  and  APB. 
The  PM  will  also  develop  and  submit  Phase  II  exit  criteria,  LRIP  quantities,  and  LRIP 
exit  criteria  for  approval  by  the  MDA.  Additionally,  the  DOT&E  and  DTSE&E  shall 
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review  and  approve  an  updated  version  of  the  PM’s  TEMP  for  all  OSD  test  and 
evaluation  related  oversight  programs.  [Ref.  5:Part  1.5.3] 

8.  Milestone  III:  Production  or  Fielding/Deployment  Approval 

This  milestone  is  the  decision  point  that  authorizes  entrance  into  the  production  of 
a  system.  The  Program  Manager  will  again  submit  updates  of  the  acquisition  strategy 
and  the  APB.  If  appropriate.  Phase  III  exit  criteria  will  be  established.  Provisions  for 
evaluation  of  post-deployment  performance  will  also  be  established  by  the  PM  for  review 
by  the  MDA.  [Ref.  5:Part  1.5.4]  With  respect  to  test  and  evaluation.  Title  10  USC2399  & 
USC2366  state  that,  “the  decision  to  proceed  beyond  LRIP  cannot  be  finalized  until  the 
DOT&E  Beyond  LRIP  and  LFT&E  reports  are  received  by  the  Congressional  Defense 
Committees.”  [Ref.  5:Part  1.5.4] 

G.  CHAPTER  SUMMARY 

As  described  in  this  chapter,  the  acquisition  management  process  model,  with  its 
milestones  and  phases,  allows  for  review  of  the  program  by  key  individuals  who  monitor 
and  administrate  the  program’s  progress,  identify  problems,  and  make  necessary 
corrections.  Combined  with  the  requirements  generation  process,  the  user  will  ultimately 
receive  a  system  that  meet  the  mission  need.  The  next  chapter  deals  specifically  with  test 
and  evaluation  and  its  role  in  the  DoD  acquisition  process. 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 
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IV.  DEPARTMENT  OF  DEFENSE  TEST  AND  EVALUATION  OVERVIEW 


A.  INTRODUCTION 

This  chapter  provides  the  reader  with  an  understanding  of  the  various  types  of  test 
and  evaluation  conducted  in  the  DoD  acquisition  cycle.  This  chapter  will  describe  the 
primary  types  of  test  and  evaluation,  T&E’s  role  in  the  acquisition  process,  T&E 
organizations  in  the  DoD  and  the  USMC,  and  lessons  learned  from  previous  testing  of 
major  acquisition  programs. 


B.  TEST  AND  EVALUATION  DEFINED 

For  acquisition  programs,  test  is  defined  as: 

Any  program  or  procedure  which  is  designed  to  obtain,  verify,  or  provide 
data  for  the  evaluation  of:  research  and  development  (other  than  laboratory 
experiments);  progress  in  accomplishing  development  objectives;  or 
performance  and  operational  capability  of  systems,  subsystems, 
components,  and  equipment  items.  [Ref.  24:p.  B-104] 

Evaluation  is  defined  as,  "the  review,  analysis,  and  assessment  of  data  obtained  from 

testing  or  other  sources."  [Ref.  25:Sect.  3.1]  When  used  together  as  a  single  concept,  test 

and  evaluation  can  be  defined  as,  "the  process  by  which  a  system  or  components  are 

compared  against  requirements  and  specifications  through  testing."  [Ref.  24 :p.  B-104] 

Thus,  test  and  evaluation  (T&E)  is,  "the  deliberate  and  rational  generation  of  data,  which 

concerns  the  nature  of  the  emerging  system,  and  the  creation  of  information  useful  to  the 

technical  and  managerial  personnel  controlling  its  development."  [Ref.  26:p.  7] 

The  purpose  of  test  and  evaluation  is  to  identify  the  areas  of  risk  in  a  defense 

system's  development  and  acquisition  program  so  that  the  risk  can  then  be  reduced  or 
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eliminated.  Decision-makers  rely  on  test  and  evaluation  results  to  verify  the  attainment 
of  technical  performance  parameters  by  the  tested  system.  Decision-makers  also  use  the 
results  to  determine  if  the  developing  system  is  operationally  effective,  operationally 
suitable,  and  survivable  in  the  environment  for  its  intended  use.  [Ref.  26:p.  5] 

C.  DEVELOPMENTAL  TEST  AND  EVALUATION 

Developmental  T&E  occurs  throughout  the  life  of  an  acquisition  system,  and  is 
generally  conducted  by  the  contractor  and  government  while  being  planned  and 
monitored  by  the  Program  Management  Office.  The  purpose  of  Developmental  test  and 
evaluation  (DT&E)  is  to,  “assist  in  engineering  design  and  development,  to  verify  that 
technical  performance  specifications  have  been  met  and  to  certify  the  system  is  ready  for 
IOT&E.”  [Ref.  27]  The  key  to  DT&E  is  its  focus  on  verification  of  the  system’s  ability 
to  meet  technical  performance  specifications.  Because  of  the  technical  focus,  DT&E 
includes  the  testing  and  evaluation  of  components,  subsystems,  preplanned  product 
improvement  changes,  and  hardware/software  integration.  [Ref.  27] 

In  the  early  phases  of  a  system’s  development,  DT&E  will  rely  on  the  use  of 
models,  simulations  and  test-beds.  As  the  system  matures,  DT&E  will  occur  using 
prototypes  or  full-scale  engineering  development  models  of  the  system  as  well  as 
production  articles  for  production  qualification  testing  and  production  acceptance  testing. 
Thus,  DT&E  supports  the  system  design  process  through  a  Simulation  Test  and 
Evaluation  Process  (STEP)  approach  as  mandated  by  the  Director,  Test,  Systems 
Engineering,  and  Evaluation  (DTSE&E).  [Ref.  27] 
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Developmental  test  and  evaluation  requires  a  close  working  relationship  between 

the  contractor  and  government  personnel.  Because  the  program  manager  remains 

ultimately  responsible  for  the  success  or  failure  of  the  program, 

The  PM  and  the  staff  test  specialists  must  foster  an  environment  that 
provides  the  contractor  with  sufficient  latitude  to  pursue  innovative 
solutions  to  technical  problems  and,  at  the  same  time,  provides  the  data 
needed  to  make  rational  trade-off  decisions  between  cost,  schedule  and 
performance  as  the  program  progresses.  [Ref.  27] 

The  contractor’s  role  in  the  total  test  program  is  significant.  Therefore,  it  is  important 

that  the  contractor  establishes  an  integrated  test  plan  early  to  ensure  that  the  scope  of  the 

contractor’s  test  program  satisfies  government  and  contractor  test  objectives.  [Ref.  27] 

D.  OPERATIONAL  TEST  AND  EVALUATION 

Operational  test  and  evaluation  (OT&E)  serves  the  primary  purpose  of 

determining  whether  systems  are  operationally  effective  and  suitable  for  the  intended  use 

by  representative  users  before  full  scale  production  or  deployment.  [Ref.  26:p.  65] 

OT&E  is  also  conducted  to  identify  needed  modifications  to  the  system,  and  to  provide 

information  on  tactics,  doctrine,  organization,  and  personnel  requirements.  [Ref.  28] 

The  key  to  OT&E  is  its  focus  on  operational  effectiveness  and  suitability.  The 

Defense  Systems  Management  College  defines  them  as  follows: 

Operational  Effectiveness:  The  overall  degree  of  mission 
accomplishment  of  a  system  when  used  by  representative  personnel  in  the 
environment  planned  or  expected  for  operational  employment  of  the 
system  considering  organization,  doctrine,  tactics,  survivability, 
vulnerability,  and  threat.  [Ref.  26:p.  57] 
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Operational  Suitability:  The  measure  of  the  ability  of  the  intended  users 
to  use  the  system  with  its  intended  support  system  and  resources.  The 
degree  to  which  a  system  can  be  placed  satisfactorily  in  field  use  with 
consideration  given  to  availability,  compatibility,  transportability, 
interoperability,  reliability,  wartime  use  rates,  maintainability,  safety, 
human  factors,  manpower  supportability,  logistics  supportability,  natural 
environment  effects  and  impacts,  documentation  and  training 
requirements.  [Ref.  26:p.58] 

Operational  test  and  evaluation  can  be  divided  into  three  general  phases: 
Operational  Assessments  (OA),  Initial  Operational  Test  and  Evaluation  (IOT&E)  and 
Follow-on  Operational  Test  and  Evaluation  (FOT&E).  Title  10,  USC  Section  2399 
mandates  that  IOT&E  be  conducted  before  proceeding  beyond  low  rate  initial  production 
(normally  Milestone  III)  to  ensure  that  a  system  has  valid  operational  effectiveness  and 
suitability.  The  law  also  requires  that  typical  users  conduct  IOT&E  on  a  system  as  close 
to  a  production  configuration  as  possible,  in  an  operationally  realistic  environment. 
Follow-on  test  and  evaluation  is  then  conducted  on  already  deployed  systems  to 
determine  if  operational  effectiveness  and  suitability  are  actually  being  attained.  [Ref.  28] 


E.  EARLY  OPERATIONAL  ASSESSMENT 

An  Early  Operational  Assessment  (EOA)  is  a  category  Of  Operational 
Assessments  (OA)  that  is  conducted  by  a  service’s  Operational  Test  Activity  (OTA) 
during  the  early  stages  of  Phase  0  (CE)  or  advanced  development  of  the  system  design 
during  Phase  I  (PDRR).  An  EOA  will  generally  evaluate  an  emerging  system’s  design 
features  by  reviewing  developmental  testing  of  brassboard  systems,  mock-ups,  or  actual 
prototypes  as  test  articles.  [Ref.  29]  An  EOA  may  make  heavy  use  of  modeling  and 
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simulation  as  an  evaluation  tool.  Although  all  Critical  Operational  Issues  (COIs)  may  be 
addressed  during  an  EOA,  COIs  are  usually  not  fully  resolved  until  later  in  the  program 
when  actual  production  representative  systems  are  available  for  the  more  comprehensive 
Initial  Operational  Test  and  Evaluation  (IOT&E).  [Ref.  29] 

The  key  element  of  an  EOA  is  its  ability  for  operational  testers  and  user 

representatives  to  identify  design  concerns  early  enough  in  the  program  to  be  cost 

effective  to  correct.  These  early  corrections  could  significantly  enhance  design  and 

reduce  total  ownership  costs  of  the  system.  Tom  Carter  of  the  Office  of  the  Director, 

Operational  Test  and  Evaluation  (DOT&E)  asserts: 

Early  involvement  of  the  Operational  testers  produces  large  payoffs  for  the 
minimal  resources  usually  required.  From  assisting  the  user  in  stating 
operational  requirements  in  a  manner  appropriate  for  the  developer  and  the 
operational  tester,  to  providing  an  independent  assessment  of  the  emerging 
system  capabilities  in  a  realistic  environment  early  enough  to  affect 
system  design  with  minimal  disruption,  the  products  gained  from  an  EOA 
usually  are  well  worth  the  cost  in  dollars  and  time.  [Ref.  29] 

Carter  further  states  that,  “A  Project  Manager’s  (PM’s)  failure  to  get  the  OTA  involved 

early  and  substantially  will  likely  have  adverse  effect  later  in  the  program,  when  the 

political  cost  could  be  great.”  [Ref.  29] 


F.  LIVE  FIRE  TEST  AND  EVALUATION 

Live  Fire  Test  and  Evaluation  (LFT&E)  is  a  significant  element  of  a  major 
defense  acquisition  system’s  pre-production  readiness  decision.  Congress  has  mandated 
(10  USC  Section  2366)  that  LFT&E  be  conducted  on  a  covered  system,  major  munitions 
program,  missile  program,  or  product  improvement  to  a  covered  system,  major  munitions 
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program,  or  missile  program  before  it  can  proceed  beyond  low-rate  initial  production. 
[Ref.  5:Part  3.4.9]  Congress  has  also  mandated  that  a  program  may  not  proceed  beyond 
low-rate  initial  production  until  realistic  survivability  or  lethality  testing  is  completed, 
and  that  the  LFT&E  report  required  by  Title  10,  USC  Section  2366  is  submitted  to  the 
appropriate  congressional  committees.  [Ref.  5:Part  3.4.9] 

The  purpose  of  LFT&E  is  to  assess  a  system’s  vulnerability/survivability  and  its 
lethality.  Survivability  is  defined  as,  “the  capability  of  a  system  and  crew  to  avoid  or 
withstand  a  man-made  hostile  environment  without  suffering  an  abortive  impairment  of 
its  ability  to  accomplish  it  designated  mission.”  [Ref.  5:App.  IV]  Though  often  used 
interchangeably,  vulnerability  is  actually  a  subset  of  survivability,  and  specifically  refers 
to  a  system’s  characteristics  that  cause  it  to  suffer  degradation  as  a  result  of  being  subject 
to  the  hostile  environment.  Lethality  is  defined  as,  “the  ability  of  a  munition  to  cause 
damage  that  will  cause  the  loss  or  a  degradation  in  the  ability  of  a  target  system  to 
complete  its  designated  mission(s).”  [Ref.  5:App.  IV]  DoD  5000.2R  further  identifies  the 
four  primary  objectives  of  LFT&E  [Ref.  5:App.  IV]: 

•  To  provide  information  to  decision-makers  on  potential  user  casualties, 
vulnerabilities,  and  lethality,  taking  into  equal  consideration 
susceptibility  to  attack  and  combat  performance  of  the  system. 

•  To  ensure  that  knowledge  of  user  casualties  and  system  vulnerabilities 
or  lethality  is  based  on  testing  of  the  system  under  realistic  combat 
conditions. 

•  To  allow  any  design  deficiency  identified  by  the  testing  and  evaluation 
to  be  corrected  in  design  before  proceeding  beyond  low-rate  initial 
production. 

•  To  assess  battle  damage  repair  capabilities  and  issues. 
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Survivability  testing  often  begins  at  the  component,  subsystem,  and  subassembly 
level  as  the  system  progresses  through  Phase  0  and  Phase  I.  Survivability  testing 
culminates  in  Phase  II  with  tests  of  the  complete  covered  system  entirely  configured  for 
combat.  [Ref.  5:Part  3.4.9]  Survivability  testing  is  complex,  and  requires  specialized  test 
facilities  that  can  measure  the  effects  of  enemy  weapons  systems  and  munitions.  Because 
to  the  nature  of  the  threat  environment,  testing  and  test  results  are  often  classified. 

G.  MULTI-SERVICE  TEST  AND  EVALUATION 

Multi-service  test  and  evaluation  is  conducted  when,  “a  system  is  determined  to 
be  of  use  to  more  than  one  service  and  acquisition  of  that  system  will  be  conducted  by 
those  services,  or  when  a  system  will  be  required  to  interface  with  equipment  of  another 
service.”  [Ref.  30:p.  22-1]  One  of  the  services  involved  in  the  acquisition  of  the  system 
will  be  designated  the  Lead  Service.  The  Lead  Service  is  then  responsible  for  the  conduct 
of  test  planning,  execution,  and  reporting.  The  Lead  Service  must  coordinate  to  ensure 
that  the  participating  services’  unique  critical  issues  and  requirements  are  being  met  with 
the  test  management  plan.  Because  of  the  potential  differing  mission  use  of  the  system, 
test  results  must  be  evaluated  in  light  of  each  services’  requirements.  [Ref.  30:p.  6-7] 

H.  CONCURRENT  AND  COMBINED  TEST  AND  EVALUATION 

Concurrent  and  combine  test  and  evaluation  are  often  confused  with  each  other  or 
are  used  synonymously.  Concurrent  test  and  evaluation  refers  to  cases  where  two 
different  types  of  testing  (i.e.  DT&E  and  OT&E)  take  place  at  the  same  time.  The  two 
tests,  however,  are  separate  and  distinct  activities.  Combined  test  and  evaluation  is  a 
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single  test  program  or  event  that  supports  the  objectives  of  the  two  different  tests.  [Ref. 
30:p.  9-1]  Table  2  outlines  stated  advantages  and  limitations  of  both  concurrent  and 
combined  testing. 


';^-^':-^'^-;.\Con»binedTesti| 

ig  and  Evaluation 

Advantages 

Limitations 

-Shortens  time  required  for  testing  and,  thus,  the 
acquisition  cycle. 

-Achieves  cost  savings  by  eliminating  redundant 

-Requires  extensive  early  coordination. 

-Test  objectives  may  be  compromised. 

activities. 

-Early  involvement  of  OT&E  personnel  during 

-Requires  development  of  DT/OT  common  test 
data  base. 

system  development  increases  their  familiarity 

-Combined  testing  programs  are  often  conducted 

with  system. 

in  a  development  environment. 

-Early  involvement  of  OT&E  personnel  permits 

-Test  will  be  difficult  to  design  to  meet  DT  and 

communication  of  operational  concerns  to 
developer  in  time  to  allow  changes  in  system 

OT  requirements. 

design. 

-The  system  contractor  is  prohibited  by  law  from 
participating  in  IOT&E. 

-Time  constraints  may  result  in  less  coverage  than 
planned  for  OT&E  objectives. 

I  Concurrent  Testing  and  Evaluation  ^  :j 

|  Advantages 

Limitations 

1  -Shortens  time  required  for  testing  and,  thus,  the 
acquisition  cycle. 

-Requires  extensive  coordination  of  test  assets. 

-If  system  design  is  unstable  and  far-reaching 

-Achieves  cost  savings  by  overlapping  redundant 

modifications  are  made,  then  OT&E  must  be  ! 

activities. 

repeated. 

I  -Provides  earlier  feedback  to  the  development 

-Concurrent  testing  programs  often  do  not  have 

process. 

developmental  test  data  available  for  OT&E 
planning  and  evaluation. 

-Contractor  personnel  frequently  perform 
maintenance  functions  in  a  DT&E.  Logistics 
support  by  user  must  be  available  earlier  for 

IOT&E. 

-Limited  test  assets  may  result  in  less  coverage 

than  planned  for  OT&E  objectives. 

Table  2.  Advantages  and  Limitations:  Combined  and  Concurrent  Testing  [Ref.  30:p.9-4] 
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L  JOINT  TEST  AND  EVALUATION 

Joint  test  and  evaluation  differs  from  multi-service  test  and  evaluation  in  that  joint 
T&E  is  a  specific  program  activity  sponsored  and  funded  by  the  OSD.  Joint  T&E 
programs  are  a  means  to  examine  joint  service  doctrine  and  tactics,  and  are  not  meant  for 
acquisition  purposes.  [Ref.  30:p.6-7] 

J.  TESTING  IN  THE  ACQUISITION  PROCESS 

The  test  and  evaluation  process  is  an  important  element  of  decision-making  in  the 
DoD  acquisition  process.  Decision-making  authorities  use  test  and  evaluation  results  to 
support  trade-off  analysis,  identify  risk  drivers,  and  assess  the  maturity  of  system 
development.  Program  Managers  develop  a  test  and  evaluation  strategy  to  ensure  that  the 
system  they  oversee  meets  the  criteria  to  advance  to  the  next  phase,  and  ultimately  to 
ensure  that  the  system  meets  the  user’s  needs.  [Ref.  30:p.  1-1]  Test  and  evaluation 
events  occurring  in  the  different  phases  will  be  described  as  they  relate  to  a  system’s 
preparation  for  milestone  review.  Figure  8  provides  a  graphic  overview  of  the  different 
test  and  evaluation  events  as  they  occur  throughout  the  DoD’s  acquisition  process. 

1.  Concept  Exploration  -  Phase  0 

Formal  test  and  evaluation  planning  begins  in  this  phase.  The  Test  and 
Evaluation  Integrated  Product  Team  (T&E  IPT)  will  form  and  begin  working  as  a  group 
to  develop  and  resolve  test  related  issues.  [Ref.  31:Chap.  4-4]  Appropriate  test  and 
evaluation  shall  be  accomplished  and  documented  to  assist  key  acquisition  personnel  in 
selecting  the  preferred  alternative  system  concepts,  and  associated  technologies  and 
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designs  for  the  proposed  systems.  The  use  of  laboratory  testing,  modeling,  and 
simulation  by  the  contractor  and  the  government  is  encouraged  in  this  phase  to  aid  in 
assessing  the  technological  feasibility  of  alternatives.  [Ref.  31:Chap.  4-4] 
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Figure  8.  Test  and  Evaluation  in  the  Acquisition  Process  [Ref.  28] 


If  a  preferred  system  concept  is  identified,  early  test  and  evaluation  will  provide 
data  for  the  concept's  evaluation  with  respect  to  potential  requirements,  tactics,  doctrine, 
organization,  training,  transportability,  and  logistic  support.  [Ref.  31:Chap.  4-4]  Testing 
in  the  CE  phase  is  generally  limited  in  order  to  devote  no  more  resources  than  necessary 
to  the  process  of  selecting  a  concept  and  to  evaluate  the  ability  of  alternative  systems  to 
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satisfy  the  MNS.  [Ref.  25: Sect.  6.2]  The  level  of  testing  in  the  CE  phase  is  also 
dependent  upon  the  test  article's  state  of  design.  Efforts  in  this  phase  may  include 
modeling  and  simulations,  testing  of  “breadboards”  and  “brassboards”,  testing  of 
components  and  subsystems,  and  proving  concepts.  Test  and  evaluation  at  this  point 
helps  to,  “identify  and  assess  high-risk  areas,  critical  components  and  subsystems; 
establish  safety  for  operational  testing;  and  assess  the  operational  impact  of  the  preferred 
concept.”  [Ref.  31:Chap.  4-4] 

The  data  and  information  obtained  from  CE  phase  testing  can  be  used  to  support  a 
go  ahead  decision  at  the  Milestone  I  review.  [Ref.  30:p.  7-1]  In  preparation  for  the 
review,  the  Program  Manager,  or  his  designated  Test  and  Evaluation  Officer,  will  be 
concerned  with  the  following  questions  [Ref  28]: 

•  What  T&E  data  exists?  Does  analysis  conclude  that  the  concept(s)  will 
work? 

•  Can  requirements,  as  stated  be  evaluated? 

•  Does  the  existing  T&E  infrastructure/technology  base  permit 
evaluation?  Approximately  what  is  the  cost  and  time  frame  to  create  the 
infrastructure/technology  base  required? 

•  How  can  T&E  favorably  impact  risk  management  or  risk  reduction? 

•  What  alternatives  exist  to  testing/test  assets?  What  are  the  risks? 

•  Has  the  preliminary  Test  and  Evaluation  Master  Plan  been  approved? 

Are  the  Critical  Operational  Issues,  minimum  acceptable  operational 
performance  requirements,  and  the  Critical  Technical  Parameters 
included? 

•  What  are  the  recommended  Phase  I  “exit  criteria”? 
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The  PM  and  T&E  Officer  will  have  ensured  that  the  following  key  documentation 
is  consistent  at  the  time  of  the  Milestone  I  review:  the  Acquisition  Decision 
Memorandum  (ADM)  (exit  criteria),  the  Operational  Requirement  Document  (ORD),  the 
Mission  Need  Statement  (MNS),  the  System  Threat  Assessment,  and  the  Test  and 
Evaluation  Master  Plan  (TEMP).  [Ref.  28] 

2.  Program  Definition  and  Risk  Reduction  -  Phase  I 

Test  and  evaluation  planning  in  the  Program  Definition  and  Risk  Reduction  Phase 
is  based  on  the  prototypes  emerging  from  the  selected  concepts.  Developmental  T&E  in 
the  PDRR  phase  is  conducted  to,  "assist  with  engineering  design,  system  development, 
risk  identification  and  to  evaluate  the  contractor's  ability  to  attain  desired  technical 
performance  in  system  specifications  and  achieve  program  objectives."  [Ref.  30:p.  1-5] 
DT&E  will  include  contractor/go vemment  integrated  testing,  engineering  design  testing, 
and  advanced  development  testing  which  usually  occurs  at  contractor  facilities.  [Ref. 
30:p.  7-3] 

Modeling  and  simulations  will  increase  in  significance  during  PDRR  as  it  will  be 
used  to  assess  areas  of  safety,  reliability,  and  other  areas  where  test  limitations  prevent, 
direct  testing  or  observation.  Modeling  and  simulations  will  also  be  used  as  a  means  to 
keep  test  costs  to  a  minimum.  [Ref.  30:p.  7-3] 

Operational  T&E  in  the  PDRR  phase  generally  consists  of  Early  Operational 
Assessments  conducted  by  a  service’s  operational  test  and  evaluation  activity.  These 
EOAs  focus  on  the  prototypes'  potential  to  meet  existing  critical  operational  issues  by 
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representative  users  in  realistic  combat  environments.  Combined  DT  and  OT  is 
encouraged  during  PDRR  as  another  means  to  reduce  test  costs  and  schedules. 

The  information  gathered  from  both  developmental  and  operational  test  and 
evaluation  in  the  PDRR  phase  contributes  significantly  to  the  Milestone  II  decision.  The 
Milestone  II  decision  will  establish  more  specific  cost,  schedule,  operational  effectiveness 
and  suitability  objectives  and  thresholds.  Documents  of  interest  to  the  T&E  Officer  at  the 
time  of  the  Milestone  II  review  include  the  ADM  (exit  criteria),  updated  TEMP,  updated 
ORD,  Acquisition  Program  Baseline  (APB),  the  early  Operational  Assessment  and  low- 
rate  initial  production  (LRIP)  guidance.  [Ref.  28]  As  with  Phase  0,  the  T&E  Officer  and 
the  T&E  IPT  will  need  to  address  the  following  questions  in  preparation  for  the 
Milestone  II  review  [Ref.  28]: 

•  Will  the  Prototype(s)  work? 

•  Can  requirements,  as  stated,  be  evaluated? 

•  Does  the  existing  T&E  infrastructure/technology  base  permit 
evaluation?  Approximately  what  is  the  cost  and  time  frame  to  create 
the  infrastructure/technology  base  required? 

•  What  are  the  capabilities/limitations  of  each  Prototype  versus  enemy 
threats? 

•  How  can  T&E  favorably  impact  risk  management  or  risk  reduction? 

What  alternatives  exist  to  testing/test  assets?  What  are  the  risks? 

•  Has  the  TEMP  been  approved?  Does  the  TEMP  contain  the 
performance  parameters  reflected  in  the  ORD,  Alternative  Analysis, 
and  APB?  Are  they  consistent? 

•  Is  each  proposed  Prototype  feasible?  Is  one  preferred? 
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•  What  are  the  recommended  Phase  II  T&E  “exit  criteria”? 

•  What  are  the  recommended  criteria  for  certification  of  readiness  for 
final  IOT&E? 

•  Does  the  Early  Operational  Assessment  (EOA)  address  the  availability 
of  and  planning  for  resources  projected  for  OT&E  test  events? 

Following  Milestone  II  approval,  program  testing  enters  the  Engineering  and 

Manufacturing  Development  Phase. 

3.  Engineering  and  Manufacturing  Development  -  Phase  II 

The  Defense  Systems  Management  College  (DSMC)  indicates  that  during  the 
EMD  Phase,  “the  selected  system  and  its  principal  items  of  support  are  fabricated  as 
engineering  development  models.”  [Ref.  28]  The  system  will  also  be  integrated,  tested, 
evaluated,  and  documented  to  assure  that  the  system’s  design  is  stable.  The  system  will 
then  be  assessed  against  the  contract  specifications  and  technical  parameters,  evaluated  as 
to  its  operational  effectiveness  and  suitability  in  realistic  combat  environments,  matched 
against  user  requirements,  and  rated  for  its  ability  to  enter  into  production.  [Ref.  3 1  :Chap. 
4-8] 

Developmental  testing  in  EMD,  “determines  whether  engineering  is  complete 
(including  design  and  maintenance  engineering),  identifies  design  problems,  recommends 
redesign,  ascertains  that  solutions  are  in  hand,  supports  decision  makers  and  provides 
recommendations  as  to  readiness  of  the  system  to  enter  OT.”  [Ref.  31:Chap.  4-8]  The 
goal  of  developmental  test  and  evaluation  in  EMD  is  to  reduce  design  risk,  support  the 
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evaluation  of  critical  technical  parameters,  and  to  ensure  contractual  compliance  by  the 
contractor.  DT&E  will  also  validate  the  general  and  detailed  specifications  and  drawings 
that  will  be  used  in  the  final  production  of  the  system.  [Ref.  31:Chap.  4-8] 

Operational  testing  and  evaluation  in  the  EMD  phase  is  very  significant.  The 
EOA  and  the  Initial  Operational  Test  and  Evaluation  (IOT&E)  are  conducted  on 
production  representative  systems,  or  on  Low  Rate  Initial  Production  (LRIP)  systems. 
EOAs  and  IOT&E  are  conducted  by  user  organizations  in  field  exercises  that  closely 
represent  combat-like  conditions.  These  fleet  units  will  assess  not  only  the  weapon 
system,  but  will  also  assess  the  doctrine,  tactics,  logistical  support,  and  organizational 
structure  necessary  to  support  the  system  in  an  operational  environment.  [Ref.  30:p.  12-5] 
The  goal  of  operational  testing  in  EMD  is  to  ensure  the  system  meets  user  requirements 
and  minimum  operational  thresholds,  and  is  ready  to  enter  full-rate  production.  [Ref. 
30:p.  12-5] 

Department  of  the  Army  (DA)  Pamphlet  73-1  describes  EMD  as  the  most  test¬ 
intensive  phase  of  the  acquisition  process.  It  further  states  that  T&E  IPTs  should  be  held 
often  to  ensure  that  test  details  are  integrated  allowing  problems  to  be  anticipated  and 
resolved  in  a  timely  manner.  [Ref.  31:Chap.  4-8]  The  PM’s  T&E  Officer  and  the  T&E 
IPT  will  need  to  address  the  following  questions  during  EMD  in  preparation  for  the 
Milestone  III  review  [Ref.  28]: 

•  For  DT&E,  what  are  the  capabilities  and  limitations  of  each  Production 
Prototype  being  developed?  What  is  the  confidence  level  in  this 
data/assessment? 
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•  For  OT&E,  are  the  Production  Prototypes  suitable  and  effective  in 
satisfying  the  mission  need?  What  is  the  confidence  level  of  this 
assessment? 

•  Have  key  performance  objectives/thresholds  been  validated  versus 
advanced  threats? 

•  Have  requirement  changes  been  incorporated  into  the  APB,  ORD,  and 
contract  specifications? 

•  Have  specification  changes  been  reflected  back  to  requirements  and 
incorporated  into  the  APB  and  ORD? 

•  Does  the  TEMP  reflect  the  changes  in  into  the  APB,  ORD,  and  contract 
specifications?  Has  it  been  approved? 

•  Should  there  be  a  major  modification  decision,  what  are  the 
recommended  Phase  III  T&E  ‘exit  criteria’? 

Coordination  among  all  agencies  involved  in  the  T&E  IPT  is  critical  because  test 

results  must  be  reviewed  by  higher  authorities  prior  to  the  Milestone  III  review. 

Specifically: 

The  director  of  OT&E  is  required  by  law  to  document  his  assessment  of 
the  adequacy  of  OT&E  and  Live  Fire  T&E,  and  the  reported  operational 
effectiveness  and  suitability  and  vulnerability/lethality  of  the  system.  This 
is  done  in  the  Beyond  LRIP  Report,  which  must  be  delivered  to  Congress 
prior  to  proceeding  beyond  LRIP.  [Ref.  28] 

The  EMD  phase  will  end  with  the  Milestone  III  review  for  a  decision  to  enter  full-rate 
production  of  the  system.  Testing  will  continue  into  the  next  phase. 


4.  Production,  Fielding/Deployment,  Operations,  and  Sustainment  —  Phase 
III 

« 

The  objectives  of  the  production,  fielding/deployment,  operations,  and 
sustainment  phase  are  to  complete  the  production  processes  once  design  stability  has  been 
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achieved,  and  to  produce  items  for  the  operational  forces.  [Ref.  28]  The  goals  for  full- 
rate  production  are,  “to  achieve  and  maintain  efficient  and  stable  production;  achieve  an 
operational  capability  that  satisfies  mission  needs;  and  conduct  an  operational  testing 
program  that  verifies  performance  and  quality,  and  corrects  deficiencies.”  [Ref.  28] 
Testing  supports  these  goals  and  objectives  by  evaluating  the  system  in  Production 
Verification  Testing  (PVT),  Production  Acceptance  Test  and  Evaluation  (PAT&E),  and 
Follow-on  Test  and  Evaluation  (FOT&E).  At  a  minimum,  T&E  will  address  the 
following  questions  [Ref.  28]: 

•  For  DT&E,  what  are  the  deployed  system’s  demonstrated  capabilities 
and  limitations?  What  are  the  capabilities  and  limitations  of 
modifications  and  upgrades? 

•  For  OT&E,  does  the  system  continue  to  be  operationally  suitable  and 
effective  in  operational  use?  Do  proposed  modifications  and  upgrades 
increase  the  suitability  and  effectiveness  of  the  system? 

•  Have  performance  objectives/thresholds  versus  advanced  threats  been 
validated?  Does  the  system  meet  these  performance 
obj  ectives/thresholds? 

•  Is  the  TEMP  current? 

The  ultimate  result  of  test  and  evaluation  in  the  PF/DOS  phase  is  the 
determination  if  the  fielded  system  is  meeting  the  user’s  needs  and  if  any  shortcomings 
and  deficiencies  need  to  be  corrected  to  improve  performance.  [Ref.  28] 


K.  TEST  AND  EVALUATION  ORGANIZATIONS 

Two  primary  offices  oversee  the  test  and  evaluation  process  in  the  OSD:  the 
Director,  Strategic  and  Tactical  Systems  (DS&TS)  and  the  Director  Operational  Test  and 
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Evaluation  (DOT&E).  Representatives  from  both  DOT&E  and  DS&TS  organizations 
actively  participate  in  program’s  integrated  product  teams  (IPT).  [Refs.  28,  105]  DoD 
test  community  restructuring  by  the  Under  Secretary  of  Defense  for  Acquisition  and 
Technology  resulted  in  the  former  Director,  Test,  Systems  Engineering,  and  Evaluation 
(DTSE&E)  being  consolidated  into  DS&TS  structure.  DS&TS  now  has  a  subordinate 
Developmental  Testing  division.  That  DT  division  carries  out  the  same  responsibilities 
as  the  previous  DTSE&E  office.  [Ref.  105]  This  change  occurred  during  the  conduct  of 
the  research,  and  where  appropriate,  references  are  still  made  to  DTSE&E. 

1.  Developmental  Test  and  Evaluation  Organizations 

The  Director,  Strategic  and  Tactical  Systems  (DS&TS)  serves  as  the  principal 
staff  assistant  and  advisor  to  the  USD(A&T)  for  developmental  test  and  evaluation 
matters,  and  has  authority  and  responsibility  for  all  DT&E  conducted  on  designated 
major  programs  and  OSD  Oversight  programs.  [Ref.  28]  Some  of  the  DS&TS’s  primary 
duties  and  responsibilities  include  [Ref.  28]: 

•  Serving  as  the  focal  point  for  coordination  of  all  major  and  OSD 
oversight  program  test  and  evaluation  master  plans  (TEMPs).  Signs  for 
approval  of  DT&E  portion  of  TEMPs. 

•  Reviewing  major  defense  acquisition  program  documentation  for 
DT&E  implications  and  resource  requirements  to  provide  comments  to 
the  USD(A&T),  DAB,  OIPT,  or  MDA. 

•  Observing  DT&E  to  ensure  adequacy  of  testing  and  to  assess  test 
results. 

•  Providing  the  DAE  and  OIPT  with  a  technical  assessment  of 
development  T&E  conducted  on  a  weapon  system. 
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•  Providing  advice  and  makes  recommendations  to  the  USD(A&T)  and 
issues  guidance  to  the  component  acquisition  executives  with  respect 
to  DT&E. 

The  Marine  Corps  Systems  Command  (MARCORSYSCOM)  is  responsible  for 
the  oversight  of  developmental  test  and  evaluation  of  Marine  Corps  systems.  The  Test 
and  Evaluation  Section  (PSE-T),  Systems  Engineering  Branch  (PSE),  Program  Support 
Directorate,  of  MARCORSYSCOM  provides  the  Program  Manager  with  assistance  in  all 
T&E  related  matters.  Specific  guidance  can  be  provided  to  the  PM  in  developing  test 
plans  and  the  Test  and  Evaluation  Master  Plan.  [Ref.  25:Sect.  2.3]  The  Marine  Corps 
Program  Managers  Test  and  Evaluation  Handbook  states  that,  “The  recurrent  and 
prominent  lessons  learned  in  past  T&E  programs  point  to  the  continuing  need  for 
coordination  between  PST-E  and  the  PM  in  the  supervision,  planning,  conduct  and 
analysis  of  major  DT&E  events.”  [Ref.  25: Sect.  5.2] 

The  Marine  Corps  is  unique  in  that  it  has  a  specialized  Amphibious  Vehicle  Test 
Branch  (AVTB)  located  in  Camp  Pendleton,  California.  AVTB  is  currently  a  subordinate 
element  of  the  Program  Management  Office  of  the  Assault  Amphibian  Vehicle  (PM 
AAV)  who  in  turn  reports  to  MARCORSYSCOM.  AVTB  provides  for  the 
developmental  testing  of  amphibious  vehicles,  and  focuses  on  waterborne  critical 
technical  parameters  such  as  water  speed  and  endurance,  seaworthiness,  surf  zone 
transitions,  and  waterborne  reliability.  [Ref.  33] 
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2.  Operational  Test  and  Evaluation  Organizations 

The  Director,  Operational  Test  and  Evaluation  (DOT&E)  establishes  department 
policies  and  procedures  for  operational  test  and  evaluation  and  live-fire  test  and 
evaluation.  [Ref.  23  :p.  10]  As  seen  in  Figure  9,  the  director  reports  directly  to  the 
SECDEF  and  has  special  reporting  requirements  to  the  Congress.  Some  of  the  specific 
duties  of  DOT&E  include  [Ref.  28]: 

•  Obtaining  reports,  information,  advice  and  assistance  as  necessary  to 
carry  out  assigned  functions  (DOT&E  has  access  to  all  records  and 
data  in  DoD  on  acquisition  programs). 

•  Signing  the  TEMPs  for  approval  of  OT&E  and  LFT&E  and  approving 
the  OT&E  funding  for  major  systems  acquisition. 

•  Approving  test  plans  on  all  major  systems  prior  to  system  starting 
OT&E  and  LFT&E  (approval  in  writing  required  before  operational 
testing  may  begin). 

•  Providing  observers  during  preparation  and  conduct  of  OT&E  and 
LFT&E. 

•  Analyzing  results  of  OT&E  and  LFT&E  conducted  for  each  major  or 
designated  defense  acquisition  program  and  submitting  a  report  to  the 
SECDEF  and  the  Congress  on  the  adequacy  of  the  test  and  evaluation 
performed. 

•  A  final  decision  to  proceed  with  a  major  program  beyond  LRIP  cannot 
be  made  until  DOT&E  has  reported  (Beyond  LRIP  and  LFT&E 
Report(s))  to  the  SECDEF  and  to  congressional  Committees  on  Armed 
Services  and  Appropriations. 

A  1997  General  Accounting  Office  (GAO)  report  assessed  the  impact  of  DoD's 
Office  of  the  Director  of  Operational  Test  and  Evaluation.  The  report  states  that  DOT&E 
oversight  of  operational  testing  and  evaluation  increased  the  probability  that  testing 
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would  be  more  realistic  and  more  thorough,  and  that  DOT&E  can  reduce  the  risk  that 
systems  are  not  adequately  tested  prior  to  the  full-rate  production  decision.  [Ref.  35:p.  3] 
The  report  further  states  that: 

Specifically,  DOT&E  was  influential  in  advocating  increasing  the 
reliability  of  the  observed  performance  and  reducing  the  risk  of  unknowns 
through  more  thorough  testing;  conducting  more  realistic  testing; 
enhancing  data  collection  and  analysis;  reporting  independent  findings; 
and  recommending  follow-on  operational  test  and  evaluation  when 
suitability  or  effectiveness  was  not  fully  demonstrated  prior  to  initiating 
full-rate  production.  [Ref.  35 :p.  3] 

The  GAO  report  concludes  that  the  effectiveness  of  DOT&E  lies  with  its  independence 
and  its  authority  to  report  directly  to  Congress  on  test  and  evaluation  matters.  [Ref.  35 :p. 

3] 

The  Marine  Corps  Operational  Test  and  Evaluation  Activity  (MCOTEA)  was 
established  on  1  April  1978  to  serve  as  the  Marine  Corps’  independent  operational  test 
activity.  [Ref.  34]  MCOTEA’s  primary  functions  is  to  ensure  that  operational  testing  for 
Acquisition  Category  (ACAT)  I,  II,  III,  and  IV(T)  programs  are  effectively  planned, 
conducted,  evaluated,  and  reported  on.  [Ref.  25:Sect.  2.3]  MCOTEA  is  also  responsible 
for  preparing  the  Test  Planning  Document  (TPD)  and  the  Detailed  Test  Plan  (DTP)  for 
operational  test  and  evaluation,  evaluating  and  analyzing  test  results,  and  preparing  the 
Independent  Evaluation  Report  (IER)  for  submission  to  the  Commandant  of  the  Marine 
Corps.  [Ref.  25:Sect.  2.3]  Other  responsibilities  include  [Ref.  34]: 

•  Developing  OT&E  policy. 

•  Chairing  and  hosting  the  Failure  Definition/Scoring  Criteria  (FD/SC)  IPT. 
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•  Reviewing  the  System’s  Operational  Requirements  Document  (ORD). 

•  Preparing  Part  IV  of  the  Test  and  Evaluation  Master  Plan. 

•  Maintaining  liaison  with  Director,  Operational  Test  and  Evaluation  (DOT&E), 
the  FMF  [Fleet  Marine  Force],  and  other  military  activities  and  commands  as 
required  for  OT&E  matters. 

L.  CONTRACTOR  TEST  AND  EVALUATION 

In  addition  to  government  test  and  evaluation,  the  contractor  plays  a  significant 
role  in  the  testing  of  a  new  system.  The  winning  contractor  is  normally  required  by  the 
Request  For  Proposal  (RFP)  to  submit  an  Integrated  Engineering  Design  Test  Plan 
shortly  after  the  contracts  initiation  for  review  by  the  PM  and  governmental  test  agencies. 
[Ref.  30:p.  7-6]  The  contractor's  test  plan  will  include  details  on  testing  against  the 
specifications,  the  Statement  of  Work  (SOW),  and  testing  of  the  contractor's  engineering 
development  and  integration  process.  [Ref.  30:p.  7-6]  DSMC’s  Test  and  Evaluation 
Management  Guide  states,  "If  the  contractor  has  misinterpreted  the  RFP  requirements  and 
the  Integrated  Engineering  Design  Test  Plan  does  not  satisfy  government  test  objectives, 
the  iterative  process  of  amending  the  contractor's  test  program  begins."  [Ref.  30:p.  7-6] 
The  process  used  to  reconcile  with  the  contractor  over  the  details  of  the  proposed  test 
plan  must  be  done  so  as  not  to  have  significant  effects  on  contract  cost,  schedule,  or 
scope,  and  must  ensure  that  the  contractor  can  meet  government  test  objectives.  [Ref. 
30:p.  7-6] 

The  PM  is  responsible  for  ensuring  that  government  oversight  of  contractor 
testing  occurs,  and  that  the  government  has  access  to  contractor  test  data,  analysis,  and 


64 


results.  Government  involvement  in  contractor  testing  ensures  that  contractor  tests  are 
reliable,  and  are  useful  in  meeting  government  test  objectives.  [Ref.  30:p.  7-5] 

M.  TEST  AND  EVALUATION  DOCUMENTATION 

This  section  provides  the  reader  with  an  understanding  of  the  five  key  test  and 
evaluation  related  documents:  the  Test  and  Evaluation  Master  Plan  (TEMP),  the  Test 
Planning  Document,  the  Detailed  Test  Plan,  the  Beyond  LRIP  Report,  and  the  Live  Fire 
Test  and  Evaluation  Report. 

1.  Test  and  Evaluation  Master  Plan 

DoD  5000.2R  directs  the  Program  Manager  to  develop  a  test  and  evaluation 
strategy  that  supports  the  program's  overall  acquisition  strategy.  The  Test  and  Evaluation 
Master  Plan  (TEMP)  is  the  required  document  that  identifies  the  overall  structure  and 
objectives  of  the  test  and  evaluation  strategy,  and  describes  the  PM's  program  to 
implement  that  strategy.  [Ref  5:App.  Ill]  The  TEMP  provides  a  framework  within 
which  to  generate  detailed  test  and  evaluation  plans,  documents  schedule  and  resource 
implications  of  the  T&E  program,  and  identifies  the  necessary  developmental  test  and 
evaluation,  operational  test  and  evaluation,  and  live  fire  test  and  evaluation  activities. 
[Ref.  5:App.  Ill] 

One  of  the  most  critical  aspects  of  the  TEMP  is  how  it  relates  program  schedule, 
test  management  strategy  and  structure,  and  required  resources  to:  Critical  Operational 
Issues  (COIs);  Critical  Technical  Parameters;  Objectives  and  Thresholds  derived  from  the 
Operational  Requirements  Document  (ORD);  Evaluation  criteria;  and  Milestone  decision 


65 


points  (Exit  Criteria).  [Ref.  5:App.  Ill]  Because  of  the  TEMP's  significance,  complexity 
and  level  of  detail,  it  should  be  developed  through  the  IPT  process.  This  allows 
MARCORSYSCOM,  MCOTEA,  MCCDC,  DOT&E,  DS&TS,  and  others  early  input  to 
the  test  and  evaluation  development  process. 

The  DOT&E  and  DS&TS  will  be  the  Office  of  the  Secretary  of  Defense  TEMP 
approval  authorities  for  an  acquisition  category  (ACAT)  I  program.  The  TEMP  for  an 
AC  AT  I  program  shall  be  submitted  to  OSD  for  approval  30  days  prior  to  the  program's 
Milestone  I  review.  TEMP  updates  are  then  required  at  each  milestone  review,  when  the 
program  baseline  has  been  breached,  or  when  the  program  has  changed  significantly. 
[Ref.  5:App.  Ill] 

2.  Test  Planning  Document 

The  Marine  Corps  PM's  T&E  Handbook  states  that,  "The  purpose  of  the  TPD  is  to 
alert  the  FMF  [Fleet  Marine  Force]  that,  sometime  in  the  future,  FMF  resources  will  be 
required  to  support  T&E  activities."  [Ref.  25:App.  B]  MCOTEA  is  responsible  for 
developing  the  TPD  as  early  as  possible  with  input  from  the  Program  Manager  so  that  the 
best  estimate  of  personnel,  equipment,  and  resources  can  be  provided  to  the  FMF.  The 
TPD  is  required  for  all  OT  and  combined  DT/OT  events.  The  PM's  T&E  Handbook 
emphasizes  that,  "The  TPD  should  be  developed  early  enough  to  permit  allocation  of 
FMF  resources  and  funding  into  the  POM  [Program  Objective  Memorandum]  cycle." 
[Ref.  25:App.  B] 
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3.  Detailed  Test  Plan 


The  purpose  of  a  Detailed  Test  Plan  (DTP)  is  to  establish  specific  instructions  and 
guidelines  for  the  conduct  of  every  phase  of  a  test  with  specific  emphasis  on  data 
collection  and  test  control.  [Ref.  25:App.  B]  DTPs  for  operational  testing  will  be 
developed  by  MCOTEA,  while  MARCORSYSCOM  and  the  PM  will  produce  DTPs  for 
government-run  developmental  testing.  Contractor  produced  DTPs  will  conform  to  the 
provisions  of  the  contract.  The  PMs  T&E  Handbook  states,  "The  size  and  complexity  of 
the  DTP  will  be  a  function  of  the  system  being  tested,  the  amount  and  adequacy  of 
previous  testing,  and  the  type  of  tests  being  conducted."  [Ref.  25:App.  B]  In  some  cases, 
the  DTP  can  span  several  volumes. 

4.  Beyond-Low-Rate  Initial  Production  (LRIP)  Report 

Before  the  Milestone  Decision  Authority  (MDA)  can  finalize  the  decision  to 
proceed  beyond  low-rate  initial  production  of  a  major  defense  acquisition  program,  Title 
10,  USC  Section  2399  requires  that  DOT&E  submit  a  report  to  the  Secretary  of  Defense, 
and  to  the  four  Defense  Committees  of  the  Senate  and  the  House  of  Representatives. 
This  "Beyond-LRIP"  report  is  prepared  by  DOT&E's  Action  Officer  responsible  for 
oversight  of  the  major  program,  and  is  based  mainly  on  the  results  of  IOT&E.  [Ref.  29] 
The  Beyond-LRIP  report  outlines  the  Director’s  official  opinion  regarding  adequacy  of 
the  OT&E  conducted  on  the  system,  and  the  operational  effectiveness  and  operational 
suitability  of  the  system  as  tested.  The  Beyond-LRIP  Report  will  also  normally  contain 
the  LFT&E  Report  required  by  Title  10,  USC  Section  2366.  [Ref.  29] 
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When  the  Beyond-LRIP  Report  nears  completion,  DOT&E  will  normally  offer 
the  draft  report  to  the  Service  for  comment.  This  provides  the  program  manager  and 
service  authorities  insight  into  what  will  be  said  about  the  program  and  its  system.  It  also 
allows  them  an  opportunity  to  attach  comments  to  the  Beyond-LRIP  report  before  it  is 
sent  to  the  Congress.  [Ref.  29] 

5.  Life  Fire  Test  and  Evaluation  Report 

DoD  5000.2R  states  that  the  results  and  overall  evaluation  of  testing  as  identified 
in  the  program's  Live  Fire  Test  and  Evaluation  strategy  will  be  documented  and 
submitted  to  DOT&E  no  later  than  120  days  after  test  completion.  An  independent  OSD 
Live  Fire  Test  and  Evaluation  Report  will  then  be  prepared  by  the  DOT&E  within  45 
days  after  receipt  of  the  service’s  Live  Fire  Test  Report.  [Ref.  5:App.  IV]  Again,  Title  10, 
USC  Section  2366  requires  that  the  Secretary  of  Defense,  or  the  DOT&E  as  delegated, 
approve  the  OSD  Live  Fire  Test  and  Evaluation  Report  and  submit  it  to  Congress  prior  to 
the  decision  to  proceed  beyond  low-rate  initial  production.  The  report  shall  address  the 
system’s  capabilities  based  on  survivability  and  lethality  testing.  [Ref.  5:App.  IV] 

N.  PREVIOUS  TEST  AND  EVALUATION  LESSONS  LEARNED 

This  section  presents  issues  and  lessons  learned  from  previous  developmental  and 
operational  testing  of  major  acquisition  programs  in  the  DoD. 

1.  Developmental  Testing  Lessons  Learned 

Captain  Arthur  J.  Aragon,  Jr.  (USA)  analyzed  key  problems  in  Developmental 
Testing  and  Evaluation  (DT&E)  in  the  United  States  Army  by  examining  seven  major 
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defense  acquisition  programs.  His  results  were  published  as  a  Master  of  Science  thesis  in 
1994  titled  A  Comparative  Analysis  of  Developmental  Test  and  Evaluation  in  the  United 
States  Army.  [Ref.  36]  Captain  Aragon  discovered  five  common  categories  of  problems 
associated  with  the  developmental  testing  of  the  seven  systems.  The  problems  and 
associated  recommendations  for  improvement  are  summarized  below. 

a.  Schedule 

Captain  Aragon  concludes  that  schedule  problems  were  the  most  common 
and  most  significant  problem  with  conducting  DT&E.  He  states,  "Schedule  problems  are 
caused  by  the  acquisition  process  which  encourages  over  optimism,  unrealistic  schedule 
estimates  and  emphasizes  completing  the  test  on  schedule  over  conducting  the  test 
according  to  plan."  [Ref.  36:p.  74]  The  following  recommendations  were  given  to 
improve  schedule  issues  [Ref.  36:p.  78]: 

•  Starting  with  the  PM  and  his  staff,  more  realistic  schedule  estimates 
should  be  made  by  all  agencies  involved. 

•  PMs  should  hold  participative  and  conclusive  TIWGs  [Test  Integration 
Working  Groups]  to  address  test  plans  and  schedules. 

•  Historical  information  and  data  from  previous  tests  should  be  used  to 
better  estimate  future  test  schedules. 

b.  Acquisition  Process 

Captain  Aragon  asserts  that  the  acquisition  process  presents  a  significant 
problem  to  conducting  DT&E  because  of  the  funding  process  and  PM  over-optimism. 
"The  funding  process  rewards  PMs  for  being  on  schedule,  under  budget  and  meeting  the 


69 


criteria  of  the  next  milestone,  but  not  for  being  critical  and  objective  about  their  system 
and  not  for  taking  a  user  perspective."  [Ref.  36:p.  75]  He  further  concludes  that  PMs  are 
generally  over-optimistic  about  planning  and  scheduling  issues.  Thus,  other  agencies  are 
forced  to  sign  up  to  unrealistic  plans  that  are  based  on  meeting  an  aggressive  schedule  not 
based  on  the  system's  readiness  for  testing.  [Ref.  36:p.  75]  His  recommendations  for 
improvement  were  [Ref.  36:p.  78]: 

•  The  analysts  should  not  promote  excessive  testing  and  should  integrate 
other  and  more  efficient  methods  of  evaluation  including  modeling  and 
simulation. 

•  Senior  decision-makers  should  reward  PMs  who  are  realistic  and 
objective  about  the  development  of  their  system. 

c.  Test  Culture 

With  respect  to  environment.  Captain  Aragon  concludes,  "that  a  negative 
test  culture  exists  and  this  culture  was  the  basis  of  many  DT&E  problems."  [Ref.  36:p. 
75]  This  negative  image  appears  to  exist  because  some  testers  and  analysts  have  earned  a 
poor  reputation  among  PMOs  by  conducting  tests  that  appeared  to  add  no  value  to  the 
process.  [Ref.  36:p.  75]  Captain  Aragon  made  the  following  recommendations  to 
improve  test  culture  [Ref.  36:pp.  78-79]: 

•  PMs  and  contractors  should  realize  the  value  that  DT&E  provides  to 
their  development  effort. 

•  Testers  should  educate  PMs  on  their  capabilities  and  demonstrate  more 
flexibility  in  packaging  test  programs. 

•  The  PM  must  make  the  testers  and  analysts  part  of  the  team  and  not 
the  bad  news  messengers. 
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•  Testers  and  analysts  should  become  more  familiar  with  the  system 
under  test  to  better  understand,  "What  to  test?" 

d.  Resource  Management 

Captain  Aragon  determined  that  improper  management  of  funding  and 
hardware  and  software  resources  lead  to  major  problems  in  DT&E.  Specifically,  "Lack 
of  funding  could  delay  test  setup,  delay  instrumentation/equipment  checks,  and  reduce 
needed  test  support  personnel."  [Ref.  36:p.  76]  He  also  concludes  that  developing 
systems  are  constrained  by  limited  prototypes,  test  models,  and  versions  of  software.  His 
recommendations  for  improving  resource  management  included  [Ref.  36:p.79]: 

•  PMs  should  plan  for  contingencies  and  not  assume  perfect  success  in 
the  test  process. 

•  PMs  should  fund  testing  to  insure  test  resources  are  available  when 
needed  for  proper  test  conduct. 

•  Tester  should  become  more  familiar  with  the  systems  under  test 
especially  software  intensive  systems. 

e.  Changes  in  Requirements 

Changing  requirements  cause  difficulties  in  defining  test  requirements, 
and  make  test  plans  and  conduct  more  difficult  and  expensive  than  originally  estimated. 
[Ref.  36:p.  76]  Changing  requirements  were  determined  to  result  from  a  lack  of 
coordination  and/or  communication  between  agencies  and  a  lack  of  understanding  of 
DT&E  processes  among  the  Combat  Developers.  This  lack  of  communication  results  in, 
"documents  such  as  the  ORD,  the  TEMP,  and  the  contract  not  matching  in  terms  of 
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requirements."  [Ref.  36:p.  76]  Captain  Aragon's  recommendations  for  improvement  were 
[Ref.  36:p.  79]: 

•  The  PM  should  insist  that  a  solid,  stable,  and  realistic  ORD  be 
maintained. 

•  The  PM  should  establish  a  better  working  relationship  among  the 
agencies  in  defining  test  requirements. 

•  The  Combat  Developer  should  appreciate  the  impact  that  changing 
requirements  has  on  the  system  and  the  test  process. 

•  The  PM  should  ensure  that  the  major  documents,  to  include  the 
contract,  are  closely  coordinated. 

As  will  be  seen,  problems  existing  in  the  developmental  testing  process 
also  occur  in  the  operational  testing  process. 

2.  Operational  Testing  Lessons  Learned 

Captain  James  B.  Mills  (USA)  analyzed  Army  weapon  system  Operational  Test 
and  Evaluations  conducted  at  Fort  Hunter-Liggett,  California  for  his  Master  of  Science 
thesis,  An  Analysis  of  Weapon  System  Readiness  for  Operational  Testing,  published  in 
1994.  [Ref.  37]  Captain  Mills  analyzed  the  AD  ATS  (LOS-F-H)  air  defense  system,  the 
Avenger  (Pedestal  Mounted  Stinger)  air  defense  system,  the  OH-58D  (AHIP)  scout 
helicopter,  and  the  Apache  (AH-64)  attack  helicopter.  Based  on  his  analysis.  Captain 
Mills  determined  that  reoccurring  problems  in  the  areas  of  test  schedules,  manuals, 
reports,  resources,  and  training,  could  have  a  significant  impact  on  a  system's  ability  to 
successful  complete  OT&E.  [Ref.  37]  Negative  OT&E  results,  or  exposed  weaknesses  in 
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a  weapon  system  during  OT&E  could  then  result  in  a  negative  Milestone  III  decision  and 
cancellation  of  the  program.  Captain  Mills  made  specific  recommendations  for  each  of 
the  five  identified  problem  areas. 

a.  Test  Schedules 

Captain  Mills  concluded  that  pre-established  schedules  drive  the  tests,  and 
not  system  readiness  for  the  testing.  "Instead  of  testing  a  system  when  it  is  ready,  the 
tendency  is  to  test  the  system  when  it  is  scheduled."  [Ref.  37:p.  71]  The  following 
specific  recommendations  for  improving  test  scheduling  were  given  [Ref.  37:p.  74]. 

•  Test  schedules  should  have  some  flexibility  to  allow  for  delays  caused 
by  training,  equipment,  instrumentation  and  weather  problems. 

•  Unplanned,  additional  testing  requirements  should  not  be  added  to  the 
test  schedule. 

•  Test  schedules  should  be  established  on  the  basis  of  systems  readiness, 
rather  than  strictly  on  milestones. 

•  Sufficient  time  should  be  planned  in  the  schedule  for  system 
maintenance  and  recovery  after  DT&E. 

b.  Technical  Manuals 

Technical  Manuals  (TMs)  are  an  integral  part  of  system/equipment 
support  requirements.  Captain  Mills  concluded  that  early  attention  to  technical  manuals 
can  result  in  a  more  accurate  product  and  led  to  fewer  logistical  support  problems  during 
OT&E.  [Ref.  37:p.  71]  His  specific  recommendation  for  improving  TMs  is,  "contractor 
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technical  writers  should  be  brought  to  the  training  and  testing  locations  to  correct  TM's  as 
problems  are  noted  by  the  users."  [Ref.  37:p.  74] 

c.  Test  Reports 

Captain  Mills  stated,  "...operational  test  reports  lack  consistency  and 
completeness  in  their  depth  of  coverage."  [Ref  37:p.  72]  The  lack  of  coverage  leads  to 
reports  that  do  not  clearly  report  what  actually  happened  in  testing,  and  limit  their 
usefulness  to  decision-makers  and  PMs.  The  specific  recommendation  for  test  reports  is 
that  they  contain  information  as  to  what  actually  happened  in  testing.  [Ref.  37:p.  74] 

d.  Test  Resources 

Test  resource  issues  where  determined  to  be  the  majority  of  the  problems 
which  occurred  during  OT&E.  [Ref.  37:p.  72]  Some  of  the  specific  recommendations  to 
improve  test  resource  issues  were  as  follows  [Ref.  37:pp.  74-75]: 

•  Sufficient  test  articles  should  be  produced  and  available  well  before 
the  operational  test  is  supposed  to  start. 

•  Systems  should  go  through  bum-in  prior  to  the  operational  test. 

•  Test  sites  should  be  adequate  in  size,  and  all  special  clearances  should 
be  obtained. 

•  Test  instrumentation  should  not  interfere  with  user  operations. 

•  All  necessary  support  equipment  should  be  available  and  operable. 

•  All  weapons  effects  simulators  should  be  tested  and  judged  to  be 
realistic  and  effective  prior  to  the  tests. 
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•  Detailed  memorandums  of  Understanding  (MOUs)  should  be  executed 
with  all  military  units  providing  test  support  personnel. 

•  OT&E  should  be  restricted  to  a  reasonable  duration. 

e.  User  Experience  Training 

Captain  Mills'  final  conclusion  was  that,  "user  experience  training  and 
testing  before  the  operational  test  is  extremely  valuable  to  the  Program  Manager  and  his 
system."  [Ref.  37:p.  73]  Early  user  training  ensures  that  problems  are  discovered  prior  to 
testing,  and  that  users  gain  realistic  experience  with  the  system  before  they  are  evaluated. 
The  specific  recommendations  to  improve  user  experience  training  were  [Ref.  37:p.  76]: 

•  The  Program  Manager  should  schedule  Force  Development  testing  and 
training  prior  to  IOT&E. 

•  Training  should  not  be  conducted  too  early,  since  there  may  not  be 
sufficient  production  representative  systems  available  to  support  the 
training,  and  users  may  forget  the  training. 

•  Prototypes  of  detailed  mock-ups  need  to  be  available  for  all  training 
conducted  before  OT&E. 

O.  CHAPTER  SUMMARY 

This  chapter  defined  test  and  evaluation,  explained  the  various  types  of  test  and 
evaluation  conducted  in  the  DoD  acquisition  cycle,  and  presented  previous  test  and 
evaluation  lessons  learned  from  other  Major  Defense  Acquisition  Programs.  The  chapter 
serves  as  background  for  the  following  chapter  on  the  AAAV  Program’s  test  and 
evaluation  strategy. 
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V.  AAAV  TEST  AND  EVALUATION  STRATEGY 


A.  INTRODUCTION 

The  purpose  of  this  chapter  is  to  establish  and  document  the  evolution  of  the 
AAAV  PMO's  test  and  evaluation  strategy  from  Milestone  0  to  the  present,  and  document 
the  anticipated  strategy  for  the  future.  The  chapter  will  begin  by  providing  an  overview 
of  the  AAAV  PMO  structure  and  its  general  acquisition  strategy  to  date.  It  will  then 
establish  the  evolution  of  the  PMO's  test  and  evaluation  strategy  by  analyzing  three 
versions  of  the  Test  and  Evaluation  Master  Plan  covering  an  eleven-year  period.  The 
chapter  will  conclude  with  an  overview  of  the  Program's  requirements  development  and 
reconciliation  process,  the  results  of  which  serve  as  a  basis  for  vehicle  performance 
evaluation.  This  chapter  provides  the  foundation  for  the  development  and  analysis  of  test 
and  evaluation  issues  faced  by  the  PMO  covered  in  the  following  chapter. 

B.  AAAV  PROGRAM  ORGANIZATION 

The  AAAV  program  was  first  initiated  in  June  1985  when  the  Naval  Sea  Systems 

Command  established  the  Marine  Corps  Assault  Amphibious  Vehicle  Program  Office 

(NAVSEA  PMS-310).  According  to  Acquisition  Plan  NAVSEA  88-043  Revision  1  (89), 

the  AAAV  program,  “...is  intended  to  design,  develop,  and  field  a  cost-effective,  state  of 

the  art  system  of  AAAV’s  to  replace  the  existing  AAV7A1  series  of  amphibians.”  [Ref. 

40:p.  1]  At  the  time  the  AAAV  was  seen  as: 

...a  high  water  speed  armored  amphibian  vehicle  capable  of  independent 
operations  in  water  and  on  land.  It  will  provide  one  of  the  principle  means 
of  tactical  surface  mobility,  armored  protection,  and  offensive  firepower 
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for  the  landing  force  during  the  ship-to-shore  phase  of  amphibious 
operations  and  subsequent  combat  operations  ashore.  [Ref.  40:p.  1] 

As  seen  in  Figure  9,  the  reporting  chain  for  the  PMS-310  AAAV  program  manager  was 

complex. 


Figure  9.  PMS-310  AAA  Program  Acquisition  Relationship.  [Ref.  41] 


In  1990,  the  Department  of  the  Navy  consolidated  various  Marine  Corps  Assault 
Amphibian  resources  from  NAVSEA  and  the  Marine  Corps  Research,  Development  and 
Acquisition  Command  (MCRDAC)  into  a  single  AAAV  program  office.  [Ref.  42:p.l] 
The  former  PMAAV  became  designated  the  Direct  Reporting  Program  Manager  (DRPM) 
for  Advanced  Amphibious  Assault  (AAA).  As  a  result,  the  DRPM  AAA  now  reports 
directly  to  the  Assistant  Secretary  of  the  Navy  (Research,  Development,  and  Acquisition), 
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“who  serves  as  the  Naval  Acquisition  Executive  (NAE),  on  all  matters  of  cost,  schedule, 
and  performance  of  assigned  programs.”  [Ref.  42:p.  4]  Command,  coordination,  and 
support  relationships  at  the  time  of  reorganization  can  be  seen  in  Figure  10.  The 
relationship  remains  the  same  with  the  exception  that  MCRDAC  is  now  designated 
Marine  Corps  Systems  Command  (MARCORSYSCOM). 


NAE 

ASN  (RD&A) 


DRPMAAA 


Prime 

Contractors 


Coordination  and  Support] 


MCRDAC 


MCCDC 

Lead 

Laboratory 

USMC  Depots 


Figure  10.  DRMP  AAA  Acquisition  Relationship.  [Ref.  42:Encl.  2] 


At  the  time  of  the  consolidation,  the  DRPM  AAA  became  responsible  for  the 
following  five  programs:  Assault  Amphibious  Vehicles  (AAV7A1  Family),  AAV 
Product  Improvements  (AAV7A1  PIP),  Advanced  Amphibious  Assault  (AAA),  Stratified 
Charged  Rotary  Engine  (SCRE),  and  Marine  Corps  Assault  Vehicles  (Engineering).  [Ref. 
42:Encl.  1]  In  1993,  the  Program  was  again  reorganized  with  the  break  out  and  shifting 
of  AAV7A1  responsibilities  to  the  PM  AAV  who  reports  to  MARCORSYSCOM.  [Ref. 
43] 
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With  respect  to  internal  organization,  the  DRPM  AAA  employs  approximately  70 
military  and  civilian  government  workers.  The  Deputy  Program  Manager,  Mr.  Walter 
Zeitfuss  (GS-15)  assists  the  DRPM,  Colonel  Blake  Robertson,  in  the  conduct  of  his 
duties.  The  program  management  office  (PMO)  is  divided  into  six  divisions  with  the 
following  supervisors  [Ref.  44]: 

•  Director,  A  AAV  Systems  (GS-15). 

•  Director  of  Engineering  (GS- 1 5). 

•  Director  of  Logistics  (GS-15). 

•  Director,  Test  and  Evaluation,  Operations,  and  Plans  (Lt  Col) 

•  Director  of  Business  and  Finance  (GS-15). 

•  Director  of  Cost  Estimation  and  Procurement  (GS- 1 4). 

A  detailed  organizational  diagram  can  be  found  in  Appendix  A. 

Following  the  Secretary  of  Defense’s  guidance  on  streamlining  acquisition,  the 
DRPM  AAA  stipulated  in  the  Product  Definition  and  Risk  Reduction  (PDRR)  Phase 
Request  for  Proposal  (RFP)  that  the  program  office  and  the  contractor  would  co-locate 
and  use  an  Integrated  Product  Process  Development  (IPPD)  approach  to  system 
development.  In  June  1996,  General  Dynamics  was  awarded  the  contract  for  the  PDRR 
phase.  They  then  purchased  and  occupied  a  facility  in  Woodbridge,  Virginia  that  now 
houses  both  the  DRPM  AAA  offices  and  the  General  Dynamics  Amphibious  Systems 
(GDAMS)  division  of  General  Dynamic  Land  Systems.  The  co-location  greatly 
facilitates  the  use  of  the  IPPD’s  Integrated  Product  Team  (IPT)  concept.  [Ref.  45:p.  30] 
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Captain  Travis  L.  Sutton  did  extensive  research  on  the  DRPM’s  use  of  IPTs  in  his 

March  1998  Master’s  Thesis  titled  Investigation  of  IPPD:  A  Case  Study  of  the  Marine 

Corps  AAAV.  In  that  thesis  he  quotes  then  DRPM,  Colonel  James  Feigley: 

My  expectation  of  collocation  was  the  absolute  need  to  cut  down  cycle 
time  from  problem  identification  to  problem  resolution...  So  by  bringing 
them  [Government  and  Contractor]  together,  my  expectation  was  to  create 
an  atmosphere  of  mutual  respect  by  being  able  to  live  and  work  together, 
and  exposing  people  to  the  realities  of  each  other’s  culture.  [Ref.  45:pp. 

30-31] 

The  reduction  in  decision  cycle  time  is  achieved  through  the  use  of  28  Program  IPTs  that 
are  formed  along  four  levels  equating  to  the  Work  Breakdown  Structure  (WBS).  Captain 
Sutton  describes  the  four  levels  as  follows  [Ref.  45:pp.  31-33]: 

•  The  “A”  level  team  deals  with  major  program  and  cost  issues  and  consists  of 
the  Executive  Management  Team  (EMT)  -  DRPM,  Vice-President  of 
GDAMS,  as  well  as  Program  Management  Team  (PMT)  level  “B”  team 
leaders. 

•  The  “B”  level  teams  (i.e.  PMTs)  are  responsible  for  project  management, 
systems  integration,  test  and  evaluation,  and  production  design.  They  maintain 
control  over  trade-off  issues. 

•  The  “C”  level  teams  monitor  and  control  discrete  performance  parameters  of 
the  vehicle,  such  as  firepower  or  mobility.  The  level  “C”  items  are  then 
delegated  down  to  the  individual  work  package  level  that  are  performed  by 
“D”  level  IPT’ s. 

Colonel  Feigley  directed  that,  “all  IPTs  are  contractor-led  with  Government 
participation.”  [Ref.  45:p.  33]  See  Appendix  B  for  a  detailed  breakdown  of  the  28  DRPM 
AAA  Program-level  IPTs. 

The  AAAV  program  oversight  is  also  conducted  through  an  IPT  process.  At  the 
highest  level  is  the  Overarching  IPT  (O-IPT).  The  O-IPT  serves  to  provide  assistance, 


81 


oversight,  and  review  while  the  AAAV  program  progresses  through  the  life-cycle 
process.  The  O-IPT  is  led  by  the  Office  of  the  Under  Secretary  of  Defense  Acquisition 
and  Technology’s  Director  of  Strategic  and  Tactical  Systems  (DS&TS)  and  his  Deputy 
Director  for  Land  Warfare.  Additional  members  include  the  DRPM,  ASN(RD&A)  and 
staff,  USD(A&T)  and  staff,  and  relevant  OSD  staff  principles.  [Ref.  46] 

Following  the  March  1995  Milestone  I  decision,  the  DRPM  AAA  initiated  an 
“Integrating”  IPT  (I-IPT)  that  served  temporarily  in  lieu  of  the  Overarching  IPT,  but  now 
serves  to  support,  “the  development  of  strategies  for  acquisition  and  contracts,  cost 
estimates,  evaluation  of  alternatives,  logistics  management,  cost-performance  trade-offs, 
etc.”  [Ref.  46]  The  purpose  of  the  Integrating  IPT  is,  “to  focus  its  efforts  on  developing 
strategies,  reviewing  program  progress,  and  identifying  and  resolving  issues.”  [Ref.  47  :p. 
2]  The  Integrating  IPT  is  chaired  by  the  DRPM  and  the  USD((A&T)DDLW)  while 
membership  includes  the  Deputy  PM,  one  ASN(RD&A)  representative,  12  OSD  Staff 
representatives  from  various  divisions,  and  one  Joint  Staff  representative.  [Ref.  47:Encl. 
2]  The  goal  of  both  the  O-IPT  and  the  I-IPT  is  to  resolve  the  majority  of  issues  at  the 
lowest  level  and  to  only  escalate  those  issues  that  require  resolution  at  a  higher  level.  The 
I-IPT  is  also  responsible  for  coordinating  the  effort  of  three  Working-IPTs  (Test  and 
Evaluation,  Cost/Performance,  and  Modeling  and  Simulation).  [Ref.  46] 

The  DRPM  AAA  has  delegated  test  and  evaluation  authority  to  his  Director,  Test 
and  Evaluation,  a  subordinate  to  the  Director,  Test  and  Evaluation,  Operations,  and  Plans. 
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By  organization,  the  T&E  Manager  has  three  subordinate  Civilian  test  billets  to  assist 
him.  The  T&E  Manager’s  duties  are  outlined  as  follows  [Ref.  78]: 

•  Responsible  for  preparing  and  coordinating  USMC,  Department  of  Navy  and 
Department  of  Defense  approval  of  the  AAAV  Test  &  Evaluation  Master  Plan 
(TEMP). 

•  Chair  of  Integrated  Process  Team  (IPT)  for  T&E  with  members  from 
MCOTEA,  MCCDC,  ASN(RD&A),  and  DoD  Test  and  Evaluation  (T&E) 
agencies. 

•  Participates  as  USMC  representative  for  all  contractor  developmental  test 
planning. 

•  Manages  Program  Office  T&E  budget  for  conduct  of  developmental  and 
operational  testing. 

•  Negotiates  support  requirements  with  DoD  Major  Range  and  Test  Base 
Facilities  (MRTBF)  where  AAAV  will  be  tested. 

•  Officer  in  Charge  (OIC)  for  9  enlisted  Marines  assigned  to  program  office  as 
test  crewmen  for  AAAV  prototypes. 

The  T&E  Manager  is  also  responsible  for  heading  the  DRPM’s  Test  and 
Evaluation  Working  IPT  (WIPT).  The  first  T&E  WIPT  was  held  on  26  March  1997  and 
discussed  the  purpose,  focus  and  intent  of  the  WIPT.  The  WIPT  also  discussed  detailed 
issues  with  respect  to  live-fire  T&E,  DT,  Safety  testing,  firepower  accuracy  testing,  and 
modeling  and  simulations.  [Ref.  48]  The  DRPM’s  Test  and  Evaluation  Master  Plan 
describes  the  T&E  WIPT’s  purpose  as,  “to  identify,  address,  communicate,  and  resolve 
interagency  test  issues  and  concerns.”  [Ref.  57]  Membership  of  the  T&E  WIPT  includes 
representatives  from  DRPM,  DS&TS,  DOT&E,  DOT&E(Live-Fire  Test),  MCCDC, 
MCOTEA,  and  AVTB.  [Ref.  51] 
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GDAMS  Test  and  Evaluation  Manager  heads  the  Program  “B”  Level  Test  and 
Evaluation  IPT.  The  Test  IPT  Charter  states,  “The  Test  and  Evaluation  IPT  will  plan, 
conduct  and  support  system,  subsystem  and  component  testing  as  necessary  to  verify 
AAAV  design  compliance  with  System/Subsystem  Specification  performance 
requirements.”  [Ref.  49]  Additionally,  the  DRPM’s  1999  Draft  TEMP  states,  “The 
Program  T&E  IPT  is  responsible  for  developing  instrumentation  plans,  detailed  test 
plans,  and  support  plans.”  [Ref.  57]  Because  test  and  evaluation  crosses  most 
organizational  lines,  GDAMS  stipulates  that  the  Test  IPT  will  consist  of  personnel,  “that 
have  a  direct  link  to  the  completion  of  tasks  associated  with  ‘validation’  of 
designs/process.”  [Ref  50:p.  5]  Presently  the  Test  IPT  includes  members  from  the 
following  sections:  DRPM  AAA,  Mobility,  C4I,  Firepower,  Integration  and  Assembly, 
Logistics,  Specialty  Engineering,  Structures  and  Auxiliary  Systems,  Test  and  Evaluation, 
Systems  Engineering,  and  Project  Management.  [Ref  52] 

While  initiating  the  Test  IPT,  GDAMS  and  the  DRPM  determined  that  the  IPT  be 
responsible  for  establishing  how  the  following  issues  would  be  handled  [Ref  50]: 

•  IPT  operation  methodology. 

•  Method  of  reporting  classified  test  information  to  pertinent  personnel. 

•  Determination  whether  System/Subsystem  Specification  N/A’s  are  truly  N/A. 

•  Determine  what  portions  of  referenced  test  documents  apply  to  AAAV 
program. 

•  Coordination  of  test  site/facility  usage. 
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•  Development  of  test  plans  and  reports. 

•  Develop  the  requirements  for  system  test  instrumentation. 

•  Support  for  AAAV  System  “Safety  Release”. 

•  Corrective  action  team  support. 

•  Dissemination,  cataloging,  and  maintaining  the  library  of  test  reports  and  data. 

•  Identify  and  monitor  the  major  risk  elements  for  test. 

•  Impacts  of  vehicle  build  options  on  the  test  program. 

•  Personnel  and  supply  support  requirements  for  test. 

•  Interfaces  with  other  government  agencies. 

•  Test  site  security  requirements. 

•  Reporting/handling  of  vendor  proprietary  test  information. 

In  addition,  the  Test  IPT  is  also  responsible  for  establishing  the  criteria  and  requirements 
for  the  following  critical  items  [Ref.  50]: 

•  Reporting  of  Safety  Incidents,  Accidents,  and  Fires. 

•  Acceptable  vehicle  down  time  during  test. 

•  Cannibalization  allowed  to  continue  test. 

•  Maintenance  activities  allowed  without  IPT  approval. 

•  Maintenance  activities  during  qualification  test  that  causes  complete  retest. 

•  Test  thresholds  needed  during  test. 

•  Decision  authority  to  Start  and  End  a  test. 


85 


•  Safety  requirements  for  test  operations  in  conjunction  with  the  testing  agency. 

•  Vendor  test  needs. 

•  Qualification  test  needs. 

•  Use  of  qualification  test  assets  to  support  system  test. 

•  Tailoring  of  tests  to  meet  program  needs. 

C.  AAAV  ACQUISITION  STRATEGY 

The  evolution  of  the  AAAV  acquisition  strategy  began  even  before  the 
establishment  of  the  PMAAV  office  PMS-3 10.  The  Marine  Corps  first  began  to  examine 
follow-on  capabilities  to  the  LVT7  in  1971.  This  led  to  the  establishment  of  the  Landing 
Vehicle  Assault  (LVA)  program  requirements  in  1973.  [Ref.  53]  The  program’s  focus 
was  on  developing  a  high-water  speed  capable  vehicle  that  could  also  function  in  high¬ 
speed  on  land.  The  program  was  cancelled  in  1979  by  the  Commandant  of  the  Marine 
Corps  for  “vulnerability,  affordability,  and  maintainability”  concerns.  [Ref.  3:pp.  21-21] 
The  lesson  derived  from  the  LVA  program  was  that,  “significant  gains  without  a  solid 
technological  foundation  leads  to  prohibitive  costs  and  unacceptable  risk.”  [Ref.  54] 

The  acquisition  of  a  replacement  vehicle  continued  with  the  selection  of  the 
Landing  Vehicle  Tracked  -  Experimental  (LVT(X))  over  the  LVA.  This  vehicle  was  a 
slow  speed  amphibian  that  was  to  meet  the  needs  of  new  requirements.  The  development 
of  the  LVT(X)  continued  until  questions  arose  about  its  ability  to  meet  even  newer 
developing  OTH  requirements.  The  program  was  cancelled  in  1985  when  the  Secretary 
of  the  Navy  determined  that,  “the  marginal  improvements  in  firepower  and  armor  in  the 
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LVT(X)  compared  with  the  LVTP7A1  were  not  worth  the  estimated  $9B  cost  of  the  new 
program.”  [Ref.  3:p.  24]  The  lesson  from  the  LVT(X)  program  was  that,  “significant 
investments  without  significant  gains  make  no  sense.”  [Ref.  54] 

The  lessons  from  these  two  program  cancellations  shaped  the  acquisition  strategy 
of  the  PMAAV.  PMS-310’s  initial  Acquisition  Plan  NAVSEA  88-043  and  the  DRPM’s 
updated  Acquisition  Plan  90-002  followed  the  traditional  acquisition  approach  of 
methodically  progressing  through  each  phase  in  preparation  for  the  next  milestone.  The 
DRPM’s  acquisition  strategy  of  1991  even  states: 


The  overall  development  strategy  for  the  AAAV  follows  the  traditional 
approach  to  the  DoD  Life  Cycle  Systems  Management  Model  in  that  all 
development  phases  will  be  executed  in  sequence  (Concept 
Exploration/Definition  (CE/D),  Concept  Demonstration/Validation 
(CD/V)  [DemVal],  and  Full  Scale  Development  (FSD)).  At  each  program 
milestone,  proposals  for  the  next  acquisition  phase  will  be  evaluated  with 
the  preferred  designs  proceeding  into  the  next  phase.  [Ref.  55] 

The  referenced  phases  equate  to  the  present  day  CE,  PDRR,  and  EMD  phases 

respectively. 

The  lessons  also  drove  the  Marine  Corps  to  establish  a  Technology  Base 

Development  Program  in  1985.  [Ref.  54]  This  Technology  Base  Development  Program 

served  to  enhance  high-water  speed  capabilities  and  reduce  risk  associated  with  those 

capabilities.  By  1990,  the  DRPM  had  began  applying  the  following  seven-step  approach 

to  the  Technology  Base  Development  Program  [Ref.  54]: 

•  Target  the  "High  Drivers"  of  affordability,  risk,  and  performance  for 
development. 
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•  Develop  each  subsystem  in  a  competitive  environment. 

•  Systematically  integrate  groups  of  technologies  in  successively  more  complex 
test  beds  -  and  test. 

•  Culminate  the  Technology  Base  Program  by  integrating  all  subsystems  into  an 
"all-up"  advanced  technology  transition  demonstrator  -  and  test. 

•  Apply  selected  Technology  Base  Developments  to  the  current  system  and  test. 

•  Do  it  all  before  deciding  upon  an  amphibious  vehicle  concept. 

•  Make  everything  (successes  and  failures)  available  to  industry  in  total  (DTIC) 
[Defense  Technical  Information  Center]. 

In  1993,  the  DRPM  AAA  convinced  the  Deputy  Secretary  of  Defense,  Dr.  Perry, 
and  the  USD(A&T)  that  an  “evolutionary  acquisition  strategy”  would  allow  the  Marine 
Corps  to  field  the  AAAV  five  years  earlier  than  the  current  Acquisition  Program  Baseline 
(APB).  [Ref.  3:pp.  66-68]  The  underlying  concepts  to  the  AAAV’s  evolutionary 
acquisition  strategy  are  analyzed  in  detail  by  LtCol  Scott  Adams  in  his  thesis,  A  Case 
Study  of  the  Advanced  Amphibious  Assault  Vehicle  (AAAV)  Program  from  a  Program 
Manager’s  Perspective  of  March  1999.  LtCol  Adams  determined  that  the  following 
program  management  office  (PMO)  decisions  have  been  critical  in  the  development  of 
the  current  acquisition  strategy  [Ref.  3:pp.  130-132]: 

•  Initiation  of  risk-reduction  projects  during  the  CE  phase,  which  resulted  in  a 
technically  complex  amphibious  vehicle  having  no  risk  area  rated  higher  than 
moderate  at  the  MS  I  Review. 

•  Conduct  of  two  Early  Operational  Assessments  during  the  CE  phase,  which 
provided  the  user  the  opportunity  to  evaluate  each  contractor’s  preliminary 
vehicle  design  very  early  in  the  design  phase. 
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•  Adoption  of  an  evolutionary  acquisition  strategy,  which  will  allow  the 
contractor  to  focus  on  developing  the  core  capabilities  of  the  AAAV.  The 
remaining  capabilities  will  be  incorporated  later  when  the  technology  is 
available. 

•  Down-selection  to  one  contractor  for  the  PDRR  phase,  which  allowed  the 
Government  to  impart  all  of  its  knowledge  on  amphibious  vehicles  to  just  one 
contractor  and  save  an  estimated  $190M  in  the  PDRR  phase. 

•  Use  of  a  CPAF  [Cost  Plus  Award  Fee]  contract  during  the  PDRR  phase, 
which  incentivizes  the  contractor  to  control  costs  and  also  provides  the  PM 
with  the  means  to  focus  the  contractor’s  developmental  efforts  on  specific 
areas  throughout  the  PDRR  phase. 

•  Inclusion  of  the  AAAV  Special  Design  Decision  provision  in  the  PDRR 
contract,  which  gives  the  contractor  an  added  financial  incentive  to  conduct 
realistic  trade  studies. 

•  Prevention  of  contractor  “buy-in”  by  showing  each  contractor  the 
Government’s  cost  estimate  on  their  proposal  for  the  PDRR  phase,  which 
resulted  in  each  contractor  increasing  their  proposed  price  by  $40M. 

•  Mandating  the  physical  co-location  of  Government  and  key  contractor 
employees,  which  allowed  the  PM  to  achieve  the  cultural  change  he  wanted 
and  to  implement  the  acquisition  reform  initiatives  he  sought. 

•  Requiring  the  use  of  IPPD  and  IPT,  which  results  in  quicker  resolution  to 
problems. 

The  most  current  acquisition  plan,  DRPM  AAA  96-1  Revision  2,  identifies  five 
acquisition-streamlining  initiatives  applicable  to  the  acquisition  strategy.  Of  the  five, 
three  represent  the  most  impact  on  the  test  and  evaluation  strategy.  The  program,  “will 
use  NDI  [Non-Developmental  Items]  as  much  as  possible  in  order  to  economize  on 
development  costs  and  take  advantage  of  economies  of  scale.”  [Ref.  56:p.  13]  The 
Demonstration  &  Validation  Phase  may  include,  “...the  acquisition  of  additional  material 
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and  services  related  to  the  D&V  phase  from  selected  D&V  contractor,  e.g.,  up  to  two 
additional  full  system  prototypes  and  related  materials  and  services.”  [Ref.  56:p.  13] 
Finally,  “...instead  of  developing  and  fabricating  a  separate  AAAV(C)  prototype,  the 
base  D&V  contract  will  require  development  and  fabrication  of  only  the  communications 
suite  for  the  AAAV(C)  variant.”  [Ref.  56:p.  13] 

As  the  DRPM  progresses  toward  his  goal  of  producing  1013  vehicles,  including 
935  personnel  carriers  and  78  command  variants,  his  mission  remains  to  “design, 
develop,  and  field  a  cost-effective,  state-of-the-art  system  to  replace  the  existing 
AAV7A1  series  of  amphibians.”  [Ref.  56] 

D.  EVOLUTION  OF  A  AAV  TEST  AND  EVALUATION  STRATEGY 

This  section  examines  the  evolution  of  the  PMO’s  test  and  evaluation  strategy 
over  an  eleven  year  period  by  drawing  primarily  from  three  Test  and  Evaluation  Master 
Plans  (TEMPs)  with  supplementary  input  from  Acquisition  Plans,  Acquisition  Program 
Baselines  (APBs),  Program  presentation  briefing  slides,  test  reports,  and  the  1995 
Integrated  Program  Summary. 

1.  Test  Management 

A  4  November  1988  draft  copy  of  the  initial  TEMP  indicates  that  test 
management  would  be  divided  into  three  areas.  First,  the  contractor  would  be  responsible 
for  the  conduct  of  their  testing  prior  to  governmental  acceptance  of  the  system.  The 
contractor’s  testing  would  be  performed  at  contractor  facilities,  “if  adequate”,  with  test 
plans  written  by  the  contractor  and  approved  by  the  government.  [Ref  58:p.  9]  PMS-3 10 
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directed  that  the  contractor  submit  recommended  Detailed  Test  and  Evaluation  Program 
Plans  for  both  Developmental  Testing  and  Operational  Testing.  [Ref.  58 :p.  9]  PMS-310 
further  directed  that  the  contractor  be  responsible  for,  “the  operations  and  all  necessary 
test  support  to  include,  but  not  be  limited  to,  maintenance,  transportation,  data,  and 
instrumentation.”  [Ref.  58 :p.  9] 

Second,  MCOTEA  was  tasked  with  responsibility  for  ensuring  that,  “operational 
testing  is  effectively  planned,  conducted  and  evaluated,”  and  for  ensuring  that  the 
Independent  Evaluation  Report  of  test  results  was  written  and  submitted.  [Ref.  58  :p.  9] 
Finally,  the  program  office  was  responsible  for,  “planning,  conducting,  and  reporting  all 
DT&E  prior  to  full  scale  production.”  [Ref.  58:p.  9] 

The  DRPM  AAA’s  initial  TEMP  was  first  submitted  for  approval  in  July  1994, 
but  was  returned  by  DOT&E  and  DTSE&E  because  of  a  Milestone  I  date  change.  The 
DRPM’s  initial  TEMP  was  finally  approved  on  6  March  1995.  The  1995  TEMP 
specifically  states  that,  “Testing  follows  a  classical  acquisition  approach  with  dedicated 
and  separate  testing  for  DT/OT.”  [Ref.  59:p,  II-l]  Though  the  TEMP  does  concede  that 
opportunities  for  concurrent  DT/OT  testing  at  remote  locations  may  exist. 

The  1995  TEMP  breaks  the  test  management  responsibilities  into  generally  the 
same  three  categories  as  the  1988  Draft  TEMP.  The  DRPM  AAA  is  responsible 
planning,  conducting,  and  reporting  all  DT&E,  while  “executing  broad  responsibility  for 
the  implementation  of  policies,  procedures  and  requirements  for  developmental  testing 
and  evaluation.”  [Ref.  59:p.  II- 1]  MCOTEA  is  still  responsible  for  the  effective  planning, 
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conduct,  and  evaluation  of  operational  testing.  The  1995  TEMP  adds  responsibility  to 
MCCDC  for  ensuring  that  input  to  the  system’s  development  comes  from  the  “User/Fleet 
Marine  Force.”  [Ref.  59:p.  II-l] 

The  contractor’s  responsibilities  again  include  the  conduct  of  acceptance  testing  at 

adequate  contractor  facilities  under  contractor  submitted  and  government  approved  plans. 

The  contractor  is  still  to  submit  recommended  T&E  Program  plans  for  DT  so  that 

unnecessary  duplicate  government  testing  does  not  occur.  However,  there  in  no  mention 

of  the  contractor  submitting  recommendations  for  OT  plans.  Other  differences  in  the 

1995  TEMP  include  more  specific  delineation  of  operation  and  maintenance  duties,  and 

test  plan  responsibilities.  Specifically, 

Contractor  and  Marine  Corps  personnel  will  participate  in  vehicle 
operation  and  system  maintenance  actions  during  all  developmental 
testing  periods  (DT-I,  DT-II,  and  IPT).  Intermediate  and  depot  level 
maintenance  support  will  be  provided  by  the  contractor  during  EOAs. 

[Ref.  59:p.  II-l] 

The  TEMP  also  states,  “DT/OT  will  be  conducted  by  the  Government  in  accordance  with 
the  Detailed  Test  Plans  (prepared  by  the  Government).”  [Ref.  59:p.  II-l] 

As  the  PMO  prepares  for  the  Milestone  II  review,  the  TEMP  is  undergoing 
revision.  The  30  August  1999  draft  version  is  considered  to  be  a  near-final  draft  with 
only  minor  editorial  corrections  remaining.  In  contrast  to  the  1995  TEMP,  the  1999  Draft 
TEMP  states,  “The  AAAV  program  will  pursue  a  streamlined  acquisition 
approach... Testing  is  being  integrated  to  the  widest  extent  possible  utilizing  a  teaming 
approach  between  the  Government  and  the  Contractor.”  [Ref.  57:p.  II-l]  Though  the 
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DRPM  retains  oversight  responsibility  and  remains  the  final  decision  authority,  both  the 
government  and  the  contractor  are  responsible  for  test  program  planning  and  execution. 
[Ref.  57:p.  II- 1]  The  intent  is  to  minimize  and  eliminate  repetitive  testing  by  both  the 
government  and  the  contractor. 

The  1999  Draft  TEMP  makes  significant  changes  that  pursue  the  DRPM’s  team 
concept.  Otherwise  distinct  duties  between  the  contractor  and  government  now  overlap, 
and  previous  government  responsibilities  now  rest  with  the  contractor.  Marine 
participation  increases  through  out  the  DT  test  phase  for  both  operations  and 
maintenance,  with  the  intent  of  the  DRPM  Marines’  participation  being  to  increase 
training  in  preparation  for  the  EOA  and  to  provide  early  and  continuous  user  feedback  on 
vehicle  performance.  DRPM  AAA  now  plans  to  use  Government  Test  Facilities  to  the 
maximum  extent  possible  where  as  contractor  owned  facilities  “may  be  considered.” 
[Ref.  57:p.  II- 1]  The  1999  Draft  TEMP  identifies  and  expands  membership  duties  and 
responsibilities  for  both  the  T&E  Working  IPT  (WIPT)  and  the  Program  T&E  IPT  (Test 
IPT).  Under  that  listing,  the  contractor,  not  the  government,  has  responsibility  for 
developing  detailed  test  plans  for  developmental  testing.  [Ref.  57:p.  II-7]  As  before, 
MCOTEA  remains  responsible  for  operational  testing.  See  Appendix  C  for  a  listing  of 
each  IPT  member’s  test  and  evaluation  responsibilities. 

2.  Concept  Exploration  Phase 

Test  and  evaluation  strategy  in  the  Concept  Exploration  Phase  was  shaped  by  two 
key  events.  First,  the  Acquisition  Decision  Memorandum  that  approved  entry  into  the  CE 
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Phase  directed  the  Program  Management  Office  to  expand  its  focus  in  the  following  three 
areas  [Ref.  60:p.  1]: 

•  Examine  all  alternatives  of  placing  infantry  ashore,  not  just  a  new  amphibious 
vehicle. 

•  Explore  standardization  with  the  Army's  Armored  Family  of  Vehicles  (AFV) 
program. 

•  Revalidate  the  Landing  Craft  Air  Cushion  (LCAC)  Acquisition  objective. 
Second,  the  Assistant  Secretary  of  the  Navy  (Research,  Development  and  Acquisition) 
recommended  in  1991,  "...that  the  program  conduct  an  independent  technical  risk 
assessment  prior  to  moving  further  through  the  Milestone  I  DAB  process,  due  to  the 
perceived  technical  risks  associated  with  a  new  AAAV."  [Ref.  61]  The  Office  of  Naval 
Research's  (ONR)  Office  of  Advanced  Technology  (OAT)  eventually  conducted 
independent  technical  risk  assessments  in  1991,  1992,  and  1994.  [Ref.  62] 

In  response  to  these  key  events,  the  PMO  established  a  two  part  CE  Phase 
Strategy.  Part  One  involved  defining  the  problem  and  exploring  alternatives.  The 
problem  was  defined  as  being  able  to,  "provide  high  speed  transport  of  embarked  Marine 
Infantry  from  ships  located  beyond  the  horizon,  to  inland  objectives...[and]  provide 
protected  land  mobility  and  direct  fire  support  during  combat  operations  ashore."  [Ref. 
63]  The  crux  of  Part  One  was  a  three-year  Cost  and  Operational  Effectiveness  Analysis 
(COEA)  of  thirteen  candidate  systems  that  eventually  determined  the  AAAV  to  be  the 
best  alternative.  [Ref.  63] 
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Part  Two  of  the  CE  Phase  strategy  was  to  exploit  the  Technology  Base  in  order  to 
reduce  the  risk  associated  with  the  preferred  alternative.  [Ref.  63]  The  test  and  evaluation 
strategy  dealt  mainly  with  Part  Two.  The  strategy  focused  on  systematically  integrating 
and  testing  groups  of  technologies  in  successively  more  complex  test  beds,  and  then 
integrating  all  subsystems  into  an  "all-up"  advanced  technology  transition  demonstrator 
for  testing.  [Ref.  54] 

Government  developmental  testing  during  this  phase  was  a  continuation  of  the 

previously  established  Technology  Base  Demonstration  Program  being  conducted  by  the 

Carderock  Division  of  the  Naval  Surface  Warfare  Center,  and  the  David  Taylor  Research 

Center.  The  testing  concept  followed  a  progression  in  technology,  size,  and  complexity. 

The  first  test  article  was  an  automotive  test  rig  (ATR)  used  for  proving  integration  of 

advanced  automotive  drive  train  concepts.  [Ref.  64]  A  0.55  scale  High  Speed 

Amphibian  and  a  High  Water  Speed  Technology  Demonstrator  (HWSTD)  were 

developed  next.  The  HWSTD  combined  automotive  and  hydrodynamic  components  into 

a  single  vehicle  in  order  to  demonstrate  the  hydrodynamic  feasibility  of  a  planing  hull 

concept.  [Ref.  65  :p.  6]  Centerline  magazine  states, 

The  17  ton  HWSTD  became  the  first  tracked  vehicle  to  attain  a  water 
speed  greater  than  25  mph... Although  not  a  full  combat  sized  vehicle..., 
the  HWSTD  proved  that  various  high  technology  components  could 
effectively  be  integrated  to  produce  a  vehicle  that  could  operate  as  a 
tracked  vehicle  on  land  and  achieve  over  20  mph  in  the  water.  [Ref.  64-p 
12] 

The  final  test  article  developed  by  the  government  was  the  Propulsion  System 
Demonstrator  (PSD).  Prior  to  PSD  fabrication,  scale  models  of  the  HWSTD  were 
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modified  to  represent  the  PSD  design  at  a  0.245  scale.  This  model  was  subject  to  several 
hundred  hours  of  testing  in  the  NSWC's  David  Taylor  Model  Basin,  Deep  Water  Carriage 
2  facility.  These  tow  tank  experiments  examined  vehicle  design  capabilities  in  both  calm 
and  sea  state  2  conditions.  [Ref.  65  :p.  8] 

The  actual  PSD  was  build  by  AAI  Corporation  and  represented  a  near-scale 
tracked  vehicle,  crewed  by  3  people,  capable  of  carrying  15  embarked  troops,  that 
performed  as  a  planing  craft  during  waterborne  operations.  The  PSD  was  propelled 
through  the  water  with  a  gas  turbine  engine  that  provided  sufficient  thrust  to  get  the 
vehicle  on  plane  after  it  deployed  a  double  angled  bow  plane  and  chine  plates  that 
covered  the  retractable  suspension  system.  [Ref.  65:p.6]  On  land,  the  bow  plane  and 
chine  plates  were  retracted,  the  suspension  system  lowered,  and  the  power  provided  by  a 
diesel  engine.  [Ref.  65:p.6] 

The  PSD  underwent  land  mode  acceptance  testing  in  May  1991  and  then 
combined  contractor  acceptance  tests  and  government  hydrodynamic  evaluations  in  June 
1991.  [Ref.  65  :p.  2]  These  initial  water  tests  were  conducted  at  the  Special  Trials  Units 
(STU)  located  at  the  Patuxent  River  Naval  Air  Station,  Maryland.  The  PSD  attained  a 
maximum  high  water  speed  of  33  mph  (28.5  knots)  at  the  STU  on  4  March  1992.  [Ref. 
65  :p.  4]  Then  in  July  1992,  the  PSD  was  shipped  to  AVTB,  Camp  Pendleton,  California 
for  hydrodynamic  testing  to,  "demonstrate  open  ocean  characteristics,  operational 
suitability  features  and  Marine  Corps  personnel  evaluations."  [Ref.  65 :p.  2]  On  17 
August  1992,  the  PSD  attained  11.5  nautical  miles  while  on  plane.  However,  in 
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September  1 992  the  PSD  failed  to  demonstrate  surf  transit  capability  because  of  seawater 
intrusion  to  electrical  boxes,  which  caused  coolant  louvers  to  open,  allowing  seawater  to 
flood  the  engine  compartment.  [Ref.  65  :p.  4]  Testing  was  suspended  until  rebuild  was 
complete  in  1993.  During  break-in  tests  of  the  rebuilt  vehicle,  a  catastrophic  failure  to 
the  turbine  power  section  occurred  which  resulted  in  cancellation  of  all  further  PSD 
testing.  [Ref.  65:p.  4]  Despite  these  setbacks,  "The  PSD  has  demonstrated  within  a  three 
year  test  effort,  the  land  mobility  equivalent  to  an  Ml  A1  Tank,  over  water  speeds  greater 
than  25  knots  and  the  internal  space/payload  required  for  carrying  troops."  [Ref.  65  :p.  2] 

Based  on  the  OAT's  risk  reduction  recommendation  to  "Demonstrate  all  high  risk 
technologies,  particularly  the  high  power  density  engine,  prior  to  full-scale  prototyping," 
DRPM  AAA  let  contracts  to  General  Dynamics  (GD)  and  United  Defense  Limited 
Partnership  (UDLP)  in  1991.  These  risk  reduction  contracts  directed  the  contractors  to 
conduct  technical  risk-reducing  experiments  and  activities  that  focused  on  all  identified 
areas  of  technical  risk  including  [Ref.  61:p.  1]: 

•  Fabrication  and  testing  of  near  full-scale  hydrodynamic  test  rigs. 

•  Weight  reduction  programs. 

•  Prototype  waterjet  fabrication  and  testing. 

•  Fabrication  and  live-fire  testing  of  armor  solutions. 

•  Extensive  vehicle  hydrodynamic  analysis. 

•  Appendage  robustness  testing  and  analysis. 
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In  1993,  the  DRPM  let  additional  contracts  to  UDLP  and  GD  for  full-scale 
automotive  test  rigs  to  evaluate  each  contractor's  AAAV  land  mobility  performance,  and 
land  mobility  subsystem  maintainability  and  durability.  [Ref.  62]  Additionally,  the  1995 
TEMP  states  that,  "A  reliable,  retractable  suspension  system  has  never  been  developed 
and  fielded  and  this  is  a  programmatic  area  of  concern."  [Ref.  59:p.  III-6]  Table  3  shows 
a  break  out  of  the  test  and  analysis  efforts  by  each  contractor  during  the  CE  Phase 
whether  completed  or  in  progress  at  the  time  of  the  Milestone  I  decision. 


United  Defense  Limited  Partnership 

General  Dynamics  Land  Systems  f 

Hydrodynamic  Test  Rig  (0.75  scale) 

Full  Scale  Appendage  Actuation 

Composite  and  Armor  Repair 

Crew  Station  Analysis 

Electronic  Motor  Efficiency/Cooling 
Hydrodynamic  Model  Testing 

Automotive  Test  Rig 

Hydrodynamic  Test  Rig  (0.8  scale) 
Hydrodynamic  Model  Testing  (1/8  scale) 
Composite  Integration 

Armor  Repair  and  Ballistic  Evaluation 
Wateijet  Design  and  Analysis 

Automotive  Test  Rig 

Table  3.  AAAV  CE  Phase  Contractor  Test  and  Analysis  Efforts.  [Ref.  59  :p.  III-3] 


The  DRPM  used  full-scale  mock-ups  of  each  contractor's  proposed  personnel 
variant  design  to  conduct  two  technical  assessments,  or  user  juries,  by  Fleet  Marine  Force 
operators  and  maintenance  personnel.  The  users  provided  insight  on  the  mock-ups 
technical,  safety,  training,  maintenance,  and  human  factor  issues  of  the  proposed  design. 
[Ref.  59:p.  III-2] 

In  addition  to  the  technical  assessments,  MCOTEA  conducted  two  EOAs  on  the 
contractor's  mock-ups.  The  EOAs  addressed  survivability,  maintainability,  human 
factors  and  safety,  and  passenger  capacity  operational  issues.  [Ref.  59:pp.  IV-3,4]  The 
1995  TEMP  provides  the  following  specifics: 
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•  Survivability.  The  EOA  partially  assessed  the  placement  of  machinery,  fuel, 
and  ammunition  to  enhance  survivability  and  personnel  protection.  The  EOA 
assessed  whether  the  AAAV(P)  presents  a  lower  silhouette/height  than  the 
AAVP7A1  for  reduced  susceptibility.  The  EOA  partially  assessed  the 
AAAV(P)'s  ability  to  generate  smoke  for  self  or  area  obscuration.  [Ref.  59:p. 
IV-3] 


•  Maintainability.  The  EOA  partially  assessed  system  design,  layout,  and  built- 
in  test/built-in  test  equipment  to  determine  if  the  AAAV(P)  exhibits  any 
detrimental  maintainability  characteristics.  [Ref.  59:p.  IV-3] 

•  Human  Factors  and  Safety.  The  EOA  partially  assessed  displays,  controls, 
equipment  design  and  layout,  crew  access,  and  safety  to  determine  if  the 
AAAV  exhibits  any  detrimental  human  factors  and  safety  characteristics. 
[Ref.  59:p.  IV-4] 

•  Passenger  Capacity.  The  EOA  fully  assessed  the  AAAV(P)'s  passenger 
capacity  to  determine  if  the  system  could  accommodate  a  reinforced  Marine 
rifle  squad  equipped  for  combat.  [Ref.  59:p.  IV-4] 

•  Other  Issues.  The  EOA  also  partially  assessed  other  operational  effectiveness 
and  suitability  issues  such  as: 

-  Crew  and  passenger  ingress/egress;  weapon(s)  field  of  fire;  and  crew 
field  of  view. 

-  Operator  training;  maintenance  personnel  training;  AAAV(P)  crew 
size;  and  special  tools/support  equipment.  [Ref.  59:p.  IV-4] 


The  DRPM's  risk  reducing  efforts  throughout  the  CE  Phase  resulted  in  the 

following  conclusions  by  the  ONP's  independent  technical  assessment  in  1992: 

The  OAT  independent  "Red  Team"  assessments  of  the  AAAV  program 
were  valuable  in  causing  programmatic  action  to  mitigate  and  eliminate 
technical  risks  in  the  AAAV  designs  and  in  the  program.  All  areas  of 
technical  risk  identified  as  "high"  have  been  reduced  to  moderate,  low  or 
have  been  eliminated.  Many  moderate  risks  have  been  reduced  to  low  or 
have  been  eliminated.  Current  assessments  are  that  the  AAAV  design  will 
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meet  all  operational  requirements  with  acceptable  levels  of  technical  risk, 
so  no  operational  requirement  revisions  are  planned.  [Ref.  61  :p.3] 

The  OAT's  1994  assessment  reached  the  same  conclusion,  and  again  no  high-risk  areas 

were  identified.  The  OAT  recommended  that  the  program  begin  Full  Scale  Prototype 

Fabrication  and  entry  into  the  next  phase.  [Ref.  62] 

In  a  memorandum  for  the  Chairman,  Conventional  Systems  Committee,  the 

Director,  Test,  Systems  Engineering,  and  Evaluation  states  that,  "Based  on  our  review  of 

the  AAAV  program  technical  status,  we  feel  it  is  ready  to  enter  the 

demonstration/validation  [PDRR]  phase."  [Ref.  66]  The  memorandum  indicates  that 

there  are  no  outstanding  issues  with  the  1995  TEMP,  that  the  program  office  has 

identified  the  key  risk  areas  and  adequately  addressed  them  in  their  test  program,  that  the 

program  has  demonstrated  satisfactory  DT  performance  to  date,  and  that  the  program  has 

adequate  DT  plans  in  place  for  the  next  phase  of  testing.  [Ref.  66:pp.  8-9]  With  respect 

to  the  test  program  the  memorandum  indicates  the  following  three  distinguishing  features 

worth  noting  [Ref.  66:p.  8]: 

•  It  makes  good  use  of  Army  combat  vehicle  technologies  including  test  ranges 
and  facilities. 

•  It  has  a  very  thorough  set  of  technical  performance  parameters  (about  60),  that 
provide  clear  performance  objectives  and  thresholds  for  each  phase  of  the  test 
program. 

•  It  provides  sufficient  time  for  all  development  testing,  and  there  is  no  test 
concurrency  with  operational  testing. 

The  DRPM's  assessment  of  the  AAAV  to  the  Acquisition  Review  Board  in  1994  was 
that,  "[The]  AAAV  is  very  technically  mature  for  the  C/E  Phase. . ."  [Ref.  67] 
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3.  Program  Definition  and  Risk  Reduction  Phase 

The  purpose  and  goals  of  the  program's  test  and  evaluation  strategy  for  the  PDRR 
Phase  remained  relatively  stable  from  1988  to  the  present.  However,  the  conduct  of  that 
strategy  was  heavily  influenced  by  the  decision  to  down-select  to  one  contractor  for  the 
PDRR  Phase,  a  revision  of  the  AAAV  Command  and  Control  Variant's  requirements,  and 
fluctuating  funding  levels. 

First,  the  DRPM’s  decision  to  down-select  to  one  contractor  in  the  PDRR  Phase 
initially  meant  that  the  conduct  of  testing  would  be  simplified  because  the  number  of  test 
articles  would  be  reduced  from  two  to  one. 

Second,  in  January  1997,  the  Commanding  General,  Marine  Corps  Combat 
Development  Command  (MCCDC)  issued  an  update  to  the  command,  control, 
communications,  computers,  and  intelligence  (C4I)  requirements  for  the  AAAV(C) 
variant.  The  updated  requirements  were  described  in  a  new  concept  of  employment.  [Ref. 
68]  The  DRPM  initially  based  his  development  plan  for  the  AAAV(C)  around  the 
requirements  to  simply  use  the  existing  AAVC7A1  communications  equipment. 
However,  the  new  concept  of  employment  called  for  a  mobile  amphibious  and  land 
forward  command  post  (CP)  capability  with  computer  workstations.  Based  on  these  new 
requirements  for  the  AAAV(C),  the  DRPM  stated,  “Changes  in  integration  requirements 
and  application  deliveries  makes  previously  planned  contract  work  on  CP 
inappropriate...  Thus  we  stopped  work  on  CP  part  of  contract.”  [Ref.  69]  The  stop  work 
period  consisted  of  three  phases.  The  first  phase  involved  deriving  and  defining 
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requirements  from  the  Concept  of  Employment.  The  second  phase  involved  assessing  the 
technical  feasibility  of  requirements.  The  final  phase  involved  determining  the  impact  of 
the  requirements  change  on  the  program’s  integration  schedule.  This  third  phase  of  stop 
work  lasted  for  approximately  1 8  months  while  the  DRPM  “waited”  on  other  program 
offices  to  complete  their  applications.  [Ref.  69] 

Third,  the  funding  levels  for  the  AAAV  program  began  to  fluctuate  significantly 
with  the  issuance  of  Program  Decision  Memorandum  (PDM)  4  in  December  1994.  The 
PDM  reduced  the  original  FY96-01  funding  stream  by  $190M  resulting  in  a  2+  year 
stretch  out  of  the  original  baseline  just  prior  to  the  Milestone  I  decision.  [Ref.  70]  This 
stretch  out  of  the  program  was  deemed  unacceptable  by  congress  and  on  30  September 
1995,  the  OSD  issued  PDM  II  which  added  back  an  additional  $107M  throughout  FY97- 
01.  The  DRPM  was  directed  to  accelerate  the  PDRR  phase  by  nine  months.  [Ref.  71  :p. 
3]  For  FY97,  the  congressional  defense  committees  appropriated  an  increase  of  $20M  to 
accelerate  the  fielding  of  the  AAAV.  [Ref.  72]  The  DRPM’s  request  for  the  funding 
stated: 


The  $20  million  FY  1997  plus-up  funding  will  be  applied  to  highly 
valuable  technical  risk  reducing  projects  that  are  not  currently  part  of  the 
baseline  Program  Definition  and  Risk  Reduction  (PDRR)  Phase  due  to  a 
previously  imposed  FY1997  funding  cap.  These  projects  include  further 
armor  development,  high  water-speed  testing,  on-the-move  weapons 
station  testing,  automotive  and  suspension  testing,  engine  development,  an 
advanced  simulation.  The  projects  will  lower  certain  technical 
uncertainties  and  mature  the  PDRR  vehicle  design,  making  the  fabrication 
and  testing  of  two  additional  prototypes  more  economically  feasible, 
allowing  concurrent  vehicle  testing  during  PDRR  and  accelerating  the 
AAAV  program  Milestone  II  DAB  review  by  4  months.  [Ref.  73] 
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The  DRPM  used  the  additional  funding  to  procure  a  second  prototype  from  the 
contractor,  and  the  Milestone  II  review  was  accelerated  from  April  2001  to  January  2001. 

In  1997,  the  DRPM  submitted  a  report  the  United  States  House  of 
Representatives’  National  Security  Committee  (NSC)  on  its  plan  to  accelerate  the  AAAV 
program.  The  accelerated  strategy  is  based  on  adding  prototypes  to  the  PDRR  testing 
effort,  initiating  AAAV  manufacturing  development  earlier,  and  creating  a  second 
delivery  of  EMD  prototypes.  [Ref.  74]  In  a  separate  request,  the  DRPM  sought 
S10.041M  for  FY98  specifically  to  add  one  prototype  to  the  PDRR  Phase  contract.  The 
justification  for  the  request  was: 

With  only  the  two  currently  planned  Demonstration  and  Validation  (D&V) 
prototypes,  developmental  testing  must  essentially  be  conducted  in  serial 
fashion,  subjecting  the  test  schedule  to  potential  risk  of  delay  if  there  are 
significant  equipment  failures.  With  an  additional  prototype,  testing 
activities  can  be  safely  conducted  in  parallel  and  the  impact  of  significant 
equipment  failures  is  mitigated.  In  addition,  three  prototypes  worth  of 
lessons  learned  from  both  their  fabrication  and  testing  will  significantly 
enhance  the  maturation  of  the  AAAV  design.  Another  important  benefit  is 
that  Government  engineers  and  Marine  operators  can  continue  to  collect 
important  reliability  data  in  parallel  with  the  independent  tester’s  Early 
Operational  Assessment.  [Ref.  75] 

The  1997  Selected  Acquisition  Report  indicates  the  program  actually  received  an  $8M 
plus-up  for  the  procurement  of  the  third  prototype.  [Ref.  76] 

In  January  1999,  the  program  office  requested  an  additional  funding  plus-up  for 
$26. 4M.  This  request  was  justified: 

...to  significantly  mitigate  technical  and  schedule  risk  and  position  the 
program  for  IOC  [Initial  Operational  Capability]  acceleration  from  FY06 
to  FY05.  This  Enhancement  will  allow  the  AAAV  Program  the 
incorporation  of  changes  into  the  AAAV  design  based  on  lessons  learned 
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from  the  Program  Definition  and  Risk  Reduction  (PDRR)  Phase 
integration  and  assembly  of  the  PDRR  prototypes,  the  PDRR  logistics 
demonstration,  and  PDRR  system  developmental  and  operational  testing. 
The  enhancement  will  also  allow... the  development  of  more  reliable 
E&MD  phase  prototypes. . . 

...[and]  will  ensure  that  the  AAAV  design  incorporates  all  required 
changes  that  address  system  hardware  and  software  reliability.  These 
reliability  changes  are  critically  important  to  successful  operational  testing 
of  the  AAAV’s  E&MD  prototypes.  [Ref.  77] 


This  plus-up  would  also  help  meet  the  PMO’s  desire  to  move  the  Milestone  II  review 
from  January  2001  to  October  2000.  [Ref.  78] 

As  will  be  seen,  even  with  the  decision  to  down-select  to  one  contractor  for  the 
PDRR  Phase,  the  revision  of  the  AAAV  Command  and  Control  Variant's  requirements, 
and  fluctuating  funding  levels,  the  overall  program’s  test  and  evaluation  strategy  for  the 
PDRR  Phase  remained  relatively  stable. 

The  1988  Draft  TEMP  outlines  the  original  objectives  for  the  program's 
Developmental  Test  and  Evaluation  efforts.  The  following  questions  were  to  be 
addressed  and  answered  over  three  Developmental  Test  events  (DT-I,  DT-II,  and  DT-III) 
[Ref.  58 :p.  14]: 

•  Is  the  AAAV  compatible  with  Naval  amphibious  warfare  ships,  MP$ 
[Maritime  Pre-positioned  Shipping]  and  MSC  [Military  Sealift  Command] 
ships? 

•  Does  the  AAAV  demonstrate  required  mobility  when  compared  to  other 
current  and  planned  combat  vehicles  with  which  it  will  operate  during 
subsequent  operations  ashore? 

•  Does  the  AAAV  demonstrate  required  waterborne  mobility  to  include 
performance  and  survival  in  specified  sea  states  and  surf? 


104 


•  Does  the  AAAV  demonstrate  the  required  capabilities  to  detect,  identify, 
acquire,  engage,  and  destroy  specified  targets? 

•  Does  the  AAAV  provide  required  survivability? 

•  Does  the  AAAV  demonstrate  required  reliability  and  maintainability  and 
durability  characteristics? 

•  Does  the  AAAV  provide  for  effective  interface  with  crew/troop/maintenance 
personnel  in  the  required  operational  environment? 

•  Does  the  AAAV  provide  for  safety  consistent  with  system  requirements? 

•  Does  the  AAAV  provide  required  ancillary  capabilities? 

•  To  what  extent  does  the  AAAV  provide  standardization  and  inter-operability 
to  maximize  commonality? 

Specifically  for  the  PDRR  Phase,  DT-I's  purpose  was  seen  as  a  means  to, 
"demonstrate  AAAV  technical  performance  characteristics,  determine  the  feasibility  of 
selected  system/subsystem  or  component  technical  design,  and  determine  the  seriousness 
of  design  risks,  logistics  supportability  and  life  cycle  costs."  [Ref.  58:p.  15]  Each  of  two 
contractors  were  to  provide  two  prototype  vehicles  for  water  mobility,  land  mobility, 
survivability,  firepower,  ballistic  protection,  and  combat  capability  performance  testing. 
Testing  by  the  contractors  was  planned  for  three  months,  while  Government  planned  DT- 
I  testing  was  scheduled  for  six  months.  [Ref.  58:pp.  15-16] 

At  the  completion  of  DT-I,  the  contractors  would  refurbish  their  vehicles,  finish 
acceptance  testing,  and  then  deliver  the  vehicle  to  the  Government  for  Operational  Test 
Event-I  (OT-I).  The  purpose  of  OT-I  would  be  to  examine  the  contractors'  vehicle  design 
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for  operational  effectiveness  and  suitability  so  that  comparisons  between  the  two  could  be 
used  for  down-selection  decisions  into  the  next  phase.  The  scope  of  OT-I  was  to 
accumulate  20  hours  per  vehicle,  per  month  for  six  months  while  operating  in  a  realistic 
environment  based  on  operational  mission  profiles.  [Ref.  58:p.  22] 

Acquisition  Plan  NAVSEA  88-043  Revision  1  (89)  specifies  only  one  prototype 
vehicle  per  contractor,  but  is  consistent  with  the  1988  Draft  TEMP  with  respect  to 
DT/OT-I's  purpose.  The  Acquisition  Plan  states,  "In  addition  to  the  design  effort 
associated  with  the  personnel  variant,  each  contractor/contractor  team  will  design 
designated  Mission  Role  Variants  (MRVs).  These  designs  together  with  DT/OT-I  test 
results  and  updated  program  documentation  will  provide  the  basis  for  transitioning  to 
Milestone  II."  [Ref.  40:p.  3]  Acquisition  Plan  DRPM-AAA  90-002  maintains  the  one 
vehicle  per  contractor  requirement  for  PDRR  Phase  testing  and  the  intent  to  use  the  test 
results  for  down-selection,  but  further  stipulates  the  development  of  an  Automotive/ 
Hydrodynamic  Test  Rig(s)  by  the  contractor.  [Ref.  79:p.  5] 

Of  note,  the  1988  Draft  TEMP,  Acquisition  Plan  NAVSEA  88-043  Revision  1 
(89)  and  Acquisition  Plan  DRPM-AAA  90-002  are  the  only  documents  found  by  the 
author  that  reference  Mission  Role  Variants.  The  1988  Draft  TEMP  lists  desired 
capabilities  for  a  personnel  mission  role  variant  (AAAV-P),  an  assault  gun  mission  role 
variant  (AAAV-AG),  a  command  mission  role  variant  (AAAV-C),  an  engineer  mission 
role  variant  (AAAV-E),  and  a  recovery  mission  role  variant  (AAAV-R).  [Ref.  58:pp.  5-6] 
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All  follow-on  documents  reference  only  the  personnel  variant  and  a  command  and  control 
variant. 

The  1995  TEMP  increases  in  detail  its  description  and  objectives  of  PDRR 
testing.  The  TEMP  describes  the  purpose  of  testing  in  this  phase  as,  "Apart  from 
demonstrating  overall  capability  as  an  amphibious  assault  system  in  a  mission  profile 
based  operational  scenario,  the  primary  purpose  of  the  D&V  prototype  will  be  to  reduce 
program  risk  through  determination  of  technical  deficiencies  in  sub-systems  and 
operational  modes  that  will  be  corrected  in  EMD."  [Ref.  59:p.  Ill- 1  ]  See  Appendix  D  for 
the  Integrated  Test  Program  Schedule  listed  in  the  1995  TEMP. 

The  purpose  of  the  1995  TEMP’s  DT-I  for  the  AAAV(P)  was  to  confirm  the 
AAAV  technical  performance  characteristics,  determine  the  feasibility  of  selected 
system/sub-system  or  component  technical  design,  determine  the  degree  of  design  risk, 
and  assess  initial  logistics  supportability  for  the  next  milestone  decision.  [Ref.  59:p.  III-7] 
The  TEMP  describes  the  prototype  to  be  tested  during  DT-I  as  a  pre-production  prototype 
that  would  be  fully  functional  for  land  and  water  mobility,  survivability, 
firepower/lethality,  and  human  factors.  [Ref.  59:p.  III-6]  Additionally,  the  prototype 
would  be  evaluated  for  its  software  maturity  and  Nuclear,  Biological,  and  Chemical 
(NBC)  collective  protection.  Throughout  DT,  the  prototype  would  undergo  Engineering 
Evaluation  Testing  at  the  highest  assembly  levels  possible.  Armor  samples  would 
continue  to  be  delivered  and  tested  by  both  the  contractor  and  the  government.  [Ref.  59:p. 
III-7] 


107 


The  1995  TEMP  identifies  the  scope  and  objectives  for  the  AAAV(C) 
communication  suite  as  evaluating  the  integration,  human  factors,  and  compatibility 
issues  associated  with  co-located  equipment  in  the  AAAV  hull.  This  evaluation  would  be 
accomplished  by  the  development  of  two  stationary  mock-ups  and  an  operational  mock- 
up  retrofit  into  a  modified  ATR,  or  ATR(C).  [Ref.  59:p.  III-9] 

The  1995  TEMP  identifies  planned  live-fire  testing  during  the  PDRR  Phase  as 
Ballistic  Hull  and  Turret  (BHT)  fabrication  and  testing,  armor  panel  testing  to  the  90% 
probability  of  no  penetration  (P^),  and  shot  line  analysis.  The  DRPM's  goal  would  be  to 
reach  a  95%  Pnp  with  the  armor  panel  testing,  and  to  evaluate  the  BHT  for  ballistic  shock, 
repairability,  and  structural  strength  against  representative  and  overmatching  threat 
systems.  [Ref.  59:p.  III-16]  "The  BHT  will  be  used  to  help  determine  the  most  efficient 
and  cost  effective  ways  to  fabricate  the  AAAV  hull  in  order  to  achieve  the  required 
ballistic  protection."  [Ref.  59:p.  III-16]  The  TEMP  estimates  436  threat  rounds  being 
fired  against  311  test  panels.  Shot  lines  analysis  and  actual  shot  data  would  be  used  to 
enhance  already  existing  live-fire  models.  [Ref.  59:p.  III-16] 

With  respect  to  operational  test  and  evaluation,  the  1995  TEMP  no  longer 
addresses  OT-I.  Instead,  the  TEMP  identifies  four  EOAs  to  be  conducted  by  MCOTEA. 
The  DOT&E  and  DTSE&E’s  TEMP  approval  letter  even  directs  the  DRPM  to  complete 
the  four  EOAs  in  time  to  support  the  Milestone  II  decision.  [Ref.  80]  EOA-1  would 
assess  an  AAAV(C)  non-operational  mock-up  for  human  factor  issues  while  focusing  on 
the  a  single  Critical  Operational  Issue  (COI);  Can  the  AAAV(C)  transport  the  Marine 
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infantry  regimental  or  battalion  commander  and  required  staff?  [Ref.  59:p.  IV-5]  EOA-2 
would  assess  a  stationary  operational  mock-up  of  the  AAAV(C)  through  the  conduct  of  a 
Command  Post  Exercise  (CPX)  by  an  infantry  battalion  commander  and  staff.  The  intent 
of  EOA-2  would  be  to  evaluate  the  communication  equipment’s  operational  effectiveness 
(cooperative  systems)  and  suitability  (logistics  support,  software,  and  reliability),  and  the 
embarked  personnel's  suitability  for  training,  human  factors,  staff  position  selection,  and 
safety.  [Ref.  59:p.  IV-5] 

EOA-3  would  be  an  extensive  one  to  three  month  assessment  of  the  AAAV(P) 
pre-production  prototype  using  amphibious  and  mechanized  operational  scenarios.  These 
scenarios  would  be  derived  from  the  established  Operational  Mission  Scenarios/Mission 
Profiles  (OMS/MP).  The  EOA  would  assess  eleven  COIs  that  focus  on  the  transportation 
of  a  reinforced  rifle  squad  from  OTH  amphibious  shipping  to  shore,  the  transition  from 
amphibious  operations  to  offensive  and  defensive  mechanized  land  operations,  and  live- 
fire  and  maneuver  exercises  on  instrumented  ranges.  [Ref.  59:pp.  IV-4  through  IV-8] 

EOA-4  would  be  an  assessment  of  the  ATR(C)'s  ability  to  command  and  control  a 
Marine  Air-Ground  Task  Force  (MAGTF)  in  a  mechanized  environment  with  supporting 
arms.  Seven  COIs  would  be  evaluated  with  similar  operational  effectiveness  and 
suitability  questions  as  assessed  in  EOA-2.  [Ref.  59:p.  IV-8] 

Both  Acquisition  Plan  DRPM  AAA  95-1  and  DRPM  96-1  Revision  2  classify  the 
program's  schedule  risk  as  "Low."  Acquisition  Plan  DRPM  AAA  95-1  allows  for  a  30- 
month  prototype  fabrication  phase,  a  six-month  combined  shakeout/acceptance  test 
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period,  an  eight-month  DT-I  period,  a  one-month  refurbishment  period,  and  a  three- 
month  OT-I  period.  [Ref.  81:p.  10]  Acquisition  Plan  DRPM  AAA  96-1  allows  for  a  15- 
month  prototype  fabrication  phase,  a  12-month  combined  shakeout/acceptance  test 
period,  an  eight-month  DT-I  period,  a  one-month  refurbishment  period,  and  a  three- 
month  OT-I  period.  [Ref.  56:p.  10]  The  two  Acquisition  Plans  boast,  "The  amount  of 
testing  allotted  is  six-to-ten  times  longer  than  conducted  in  D&V  during  the  X-Ml  tank 
program."  [Refs.  56&70:p.  10]  Acquisition  Plan  DRPM  AAA  96-1  also  indicates  that  the 
eight-month  DT-I  period  attempts  to  accommodate  prevailing  ocean  weather  patterns  at 
AVTB,  Camp  Pendleton,  California.  [Ref.  56:p.  10] 

The  1999  Draft  TEMP  identifies  significant  changes  to  the  conduct  of 
developmental  testing  and  EOAs  in  the  PDRR  Phase.  Because  of  the  system's  maturity 
exiting  the  CE  Phase,  DT-I's  testing  goal  became,  “to  produce  an  extremely  mature 
AAAV  design”  by  testing  three  prototypes'  land  mobility,  track  robustness,  bollard  pull, 
planing  thresholds,  reverse  speeds,  at-sea  maneuverability,  ride  quality  in  3-foot  seas, 
stopping  distances,  towing  capabilities,  and  surf  zone  negotiation.  [Ref.  57:p.  III-l]  DT-I 
test  objectives  to  meet  the  extreme  maturity  goal  include  the  following  [Ref.  57:p.  5]: 

•  Determine  the  system's  capabilities  to  demonstrate  Milestone  II  exit  criteria. 

•  Determine  the  system's  potential  to  satisfy  critical  technical  parameters. 

•  Detect  any  safety  design  deficiencies. 

•  Provide  data  to  evaluate  current  software  and  design  maturation  and  provide 
input  to  design  initiatives  supporting  DT-II  phase. 
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•  Certify  readiness  for  EOA. 

The  TEMP  also  states,  "Testing  will  conform  with  the  requirements  of  DT-I...for 
baseline  testing,  however  the  vehicles  will  be  tested  to  the  edge  of  the  design  envelope  as 
defined  by  the  ORD."  [Ref.  57:p.  III-8]  As  discussed  in  the  Test  Management  section, 
DT-I  will  be  conducted  as  an  integrated  effort  between  the  contractor  and  the 
government.  MCOTEA  will  also  maximize  integration  of  DT  and  EOA  events  to  reduce 
and  avoid  duplication  of  efforts.  See  Appendix  E  for  the  Integrated  Program  Schedule 
listed  in  the  1 999  Draft  TEMP. 

The  PDRR  Phase  test  schedule  becomes  increasingly  complex  with  the  addition 
of  two  more  prototypes.  The  current  plan  is  for  the  three  AAAV(P)  prototypes  to 
undergo  separate  test  events  as  they  come  on-line  so  that  the  total  time  spent  in  the  PDRR 
Phase  is  reduced.  Shakeout  and  test  site  locations  to  complete  the  schedule  originally 
included  Quantico  and  Ft.  Story,  Virginia;  Pax  River  Naval  Air  Station  (NAS)  and 
Aberdeen  Test  Center  (ATC),  Maryland;  Eglin  Air  Force  Base,  Florida;  and  AVTB, 
Camp  Pendleton,  California.  [Ref.  57:p.  III-6]  The  additional  prototypes  also  allow  for 
the  conduct  of  a  logistics  demonstration  in  the  PDRR  Phase.  The  logistics  demonstration 
will  serve  to  identify  maintainability  deficiencies,  verify  that  supportability  requirements 
have  been  met,  and  allow  Fleet  Marine  Force  personnel  to  provide  input  for  corrective 
actions  in  the  next  phase.  [Ref.  57:p.  III-9] 

The  conduct  of  previously  planned  EOAs  changed  in  the  1999  TEMP  based  on 
the  AAAV(C)  stop- work  action.  EOA-2  and  EOA-4  were  cancelled,  while  EOA-1  was 
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modified  in  location  and  scope,  and  renamed  EOA  AAAV(C).  The  objectives  of  the 
AAAV(C)  non-operational  mock-up  EOA  are  now  [Ref.  57:p.  IV- 10]: 

•  To  evaluate  Human  Factors  including  the  man-machine  interface, 
collaborative  planning  capability  and  placement/layout  of  workstations. 

•  To  evaluate  the  suitability  of  the  workstation  to  support  effective  staff 
functioning. 

•  To  evaluate  the  ability  of  the  staff  to  operate  the  workstations  in  crowded 
conditions. 

•  Identify  user-originated  recommendations  for  improving  the  system  design. 

•  Identify  needed  improvements  in  procedures,  instrumentation,  and  resources 
for  future  Operational  Tests. 

EOA-3  remains,  but  becomes  referred  to  as  the  AAAV(P)  Pre-production  Prototype  EOA 
with  the  following  objectives  [Ref.  57:p.  IV-5]: 

•  To  estimate  the  AAAV(P)'s  potential  to  be  operationally  effective  and 
suitable. 

•  Provide  an  early  assessment  of  force-level  effectiveness  based  on 
demonstrated  system  performance. 

•  Provide  AAAV  system  performance  data  in  support  of  Modeling  and 
Simulation  efforts. 

•  Identify  user-originated  recommendations  for  improving  the  system  design. 

•  Assess  system  adequacy  relative  to  the  Concept  of  Employment. 

•  Provide  information  on  AAAV  waterborne  movement  and  command  and 
control  in  support  of  Ship-To-Objective-Maneuver  (STOM). 

•  Identify  safety  risks  and  assess  mitigation  procedures. 
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Identify  needed  improvements  in  procedures,  instrumentation,  and  resources 
for  future  Operational  Tests 


The  AAAV(P)  EOA  will  be  conducted  in  three  parts.  First,  the  vehicle  will  be 
assessed  for  its  ability  to  conduct  amphibious  operations  while  completing  designated 
Operational  Mission  Profiles  (OMPs)  at  Camp  Pendleton,  California.  Second,  the 
vehicle’s  Mk  44  Automatic  Cannon  and  Mk  46  Weapon  Station  will  be  assessed  for  their 
ability  to  conduct  live-fire  and  maneuver  at  both  Camp  Pendleton  and  the  Marine  Corps 
Air-Ground  Combat  Center  (MCAGCC),  29  Palms,  California.  The  third  phase  will 
assess  the  vehicle’s  ability  to  conduct  unrestricted  land  mobility  and  mechanized 
operations  in  a  desert  and  hot  weather  environment.  [Ref.  57:pp.  IV-5  though  IV-7]  The 
TEMP  indicates  that,  “This  preliminary  assessment  of  AAAV(P)  operational 
effectiveness  and  suitability  will  support  the  Milestone  II  decision...”  [Ref.  57:p.  IV-5] 

The  1999  Draft  TEMP  significantly  expands  the  level  of  detail  and  objectives  for 
the  Live-Fire  Test  and  Evaluation  (LFT&E)  portion  of  Section  IV.  The  PMO’s  LFT&E 
program  is  managed  by  a  separate  LFT&T  IPT  responsible  for,  “LFT&E  planning, 
program,  execution  and  management,  assessment  and  reporting  of  program  results.”  [Ref. 
57:p.  IV- 17]  The  AAAV  LFT&E  strategy  provides  for  the  “comprehensive  evaluation 
of  AAAV  vulnerability  and  lethality”  with  a  stated  goal  of  identifying  “potential 
susceptibilities  and  vulnerabilities  as  early  in  the  development  cycle  as  possible  so  that 
the  most  survivable  AAAV  possible  can  be  fielded.”  [Ref.  57:p.  IV- 17] 

The  LFT&E  strategy  follows  a  building  block  approach  by  proceeding  from 
armor  and  component  ballistic  tests,  to  ballistic  hull  and  turret  (BH&T)  testing,  to  full-up 
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system  level  (FUSL)  testing.  The  testing  uses  extensive  modeling  and  simulation  to 
augment  testing  and  estimate  expected  vulnerability  performances.  [Ref.  57:p.  IV-17] 
Specific  PDRR  Phase  LFT&E  includes  component  ballistic  testing,  armor  ballistic 
characterization  testing,  controlled  damage  testing,  and  commencement  of  BH&T  testing. 
[Ref.  57:p.  II-8]  See  Appendix  F  for  the  Live-Fire  Schedule  listed  in  the  1999  Draft 
TEMP. 


Lethality  testing  will  include  testing  and  experimentation  to  evaluate  30mm  High 
Explosive  (HE)  ammunition  for  inclusion  in  the  Joint  Munition  Effectiveness  Manual 
(JMEM).  The  DRPM  AAA,  Navy,  and  Air  Force  will  conduct  these  experiments  and 
Arena  tests  to  develop  a  performance  specification  for  joint  requirements  in  order  to  field 
a  common  DoD  family  of  30mm  ammunition.  [Ref.  57:p.  IV-18] 

Other  PDRR  Phase  testing  events  addressed  in  the  1999  Draft  TEMP  include 
[Ref.  57]: 

•  Lethality  trade-off  studies  resulting  in  the  selection  of  the  30mm  Mk  46 
Weapon  Station. 

•  Foreign  Competitive  Testing  (FCT)  by  Germany,  the  United  Kingdom,  and 
Norway  for  vehicle  components  and  30mm  ammunition. 

•  Modeling  and  simulations  to  evaluate  two  Reliability,  Availability,  and 
Maintainability  (RAM)  critical  technical  parameters  (CTPs)  that  lack 
sufficient  data  input  from  DT-I  (Mean  Time  Between  Critical  Mission  Failure 
(MTBCMF)  and  Operational  Availability  (A0)). 
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4.  Engineering  and  Manufacturing  Development  Phase 

The  test  and  evaluation  strategy  in  the  EMD  Phase  has  been  shaped  by  both  the 
funding  changes  and  the  AAAV(C)  requirements  change.  As  with  the  PDRR  Phase,  the 
planned  conduct  of  the  test  and  evaluation  strategy  has  evolved  while  the  strategy’s 
objectives  remain  relatively  stable. 

Funding  level  changes  led  to  the  program  adopting  an  accelerated  schedule 
causing  the  EMD  dates  to  shift  forward  by  almost  two  years.  The  1995  TEMP’s 
integrated  test  program  schedule  shows  EMD  starting  in  the  second  quarter  of  FY02  and 
ending  at  the  close  of  FY07.  [Ref.  59:p.  II-2]  The  1999  Draft  TEMP’s  integrated 
program  schedule  shows  the  EMD  phase  beginning  in  the  first  quarter  of  FY01  and 
ending  in  the  third  quarter  of  FY05.  [Ref.  57:p.  II-8] 

The  program’s  EMD  test  and  evaluation  strategy  begins  with  the  1988  Draft 
TEMP  identifying  a  second  Developmental  Test  event  occurring  in  the  EMD  Phase.  The 
DT-II  event  would  occur  in  two  parts.  The  first  part  would  use  refurbished  PDRR 
prototypes,  while  the  second  part  would  test  newer  more  mature  prototypes  and  Mission 
Role  Variants.  [Ref.  58:p.  17]  The  purpose  of  DT-II  is  described  as  [Ref.  58:p.  17]: 

•  Demonstrating  that  engineering  is  reasonably  complete. 

•  Ensuring  all  significant  design  problems  that  affect  compatibility,  reliability, 
maintainability,  and  logistic  considerations  have  been  identified,  and  that 
solutions  are  developed  and  economically  feasible. 

•  Demonstrating  R&M  performance  characteristics,  terrain  trafficability, 
environmental  compatibility,  and  Integrated  Logistic  Support  (ILS) 
requirements. 
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•  Demonstrating  that  technical  risks  have  been  minimized. 

•  Evaluating  training  programs. 

Acquisition  Plan  NAVSEA  88-043  indicates  that  the  single  contractor  for  this 
phase  will  develop  prototypes  for  both  developmental  and  operational  testing  to  prove 
that,  “the  design  and  the  related  support  equipment,  facilities  and  software  meet  all 
required  specifications.”  [Ref.  40:p.  5]  The  Acquisition  Plan  does  not  designate  the 
number  of  prototypes  to  be  built  in  EMD  and  in  fact  states,  “...development  and  testing 
of  a  number  of  AAAV  prototype  vehicles  limited  to  the  quantity  necessary...”  [Ref.  40:p. 
8]  Acquisition  Plan  DRPM-AAA  90-002  refers  to  vehicle  quantities  as,  “.. .up  to  25  FSD 
[Full  Scale  Development]  prototype  AAAVs.”  [Ref.  79 :p.  5] 

The  1995  TEMP  specifies  that  ten  production  representative  AAAV(P)  prototypes 
and  one  production  representative  AAAV(C)  prototype  be  manufactured  in  the  EMD 
phase.  The  eleven  prototypes  are  to  undergo  DT-II  testing  for  approximately  twenty- 
three  months  while  accumulating  between  20  and  50  hours  of  operations  per  month.  DT- 
II  would  be  conducted  by  the  government,  at  government  facilities,  and  supported  by  the 
contractor.  [Ref.  59:p.  III-l  1]  Additionally,  eleven  prototypes  would  allow  for  individual 
vehicle  testing  at  separate  locations,  and  would  allow  for  combined  vehicle  operations  at 
the  assault  amphibian  platoon  and  section  level.  [Ref.  59:p.  III-l  1] 

The  1995  TEMP  states  that: 

DT-II  testing  is  designed  to  demonstrate  that  engineering  is  reasonably 
complete,  that  technical  risks  have  been  minimized,  that  critical 
characteristics  have  been  achieved  (if  not  achieved,  then  technical 
solutions  have  been  identified),  to  support  a  Low  Rate  Initial  Production 
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and  Full  Rate  Production  decision.  Critical  objectives  to  meet  during 
testing  will  again  include  mobility  (land  and  water),  lethality,  and 
survivability  requirements.  [Ref.  59:p.  III-10] 

Tests  designated  to  occur  in  DT-II  would  involve  speed,  range,  navigation, 

communication,  ride  quality,  human  factors  and  safety,  amphibious  ship  compatibility, 

transportability,  target  acquisition,  weapon  lethality,  and  mobility.  Very  specific  testing 

would  include  surf  testing,  cold  environment  testing,  nuclear  and  NBC  effects  testing,  hot 

weather/tropics  testing,  EMI  testing,  RAM-D  testing,  and  ILS  requirements  evaluations. 

[Ref.  59:p.  Ill- 10] 

Acquisition  Plan  DRPM-AAA  95-1  indicates  that  DT-II  and  Initial  Operational 
Test  and  Evaluation  (IOT&E)  are  planned  for  the  EMD  Phase  using  LRIP  vehicles.  [Ref. 
81:p.  17]  Acquisition  Plan  DRPM-AAA  96-1  Revision  2  states  that  DT-II  and  IOT&E 
are  still  planned  for  EMD,  but  the  events  would  use  “production  representative  vehicles” 
vice  LRIP  vehicles.  [Ref  56:p.  17] 

The  1999  Draft  TEMP  describes  extensive  changes  to  the  test  concept  for  the 
AAAV(C)  variant.  At  the  completion  of  AAAV(P)  DT-I,  one  prototype  will  be 
converted  into  an  AAAV(C)  in  Fiscal  Year  (FY)  01,  and  then  begin  a  separate 
development  test  plan  track.  The  objectives  of  the  AAAV(C)  DT-I  include  determining 
the  system's  potential  to  satisfy  Critical  Technical  Parameters  (CTPs),  determining 
human  factor  issues,  identifying  levels  of  design  risk,  and  ensuring  that  the  system’s 
technical  maturity  is  sufficient  to  support  OT  certification  prior  to  Operational 
Assessment  (OA)  I.  The  focus  will  be  on  system,  sub-system  and  inter-operability 
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testing.  [Ref.  57:p.  III-l  1]  DT-II  for  the  AAAV(C)  will  be  off-set  from  AAAV  (P)  DT- 
II,  but  will  meet  the  same  objectives. 

The  1999  Draft  TEMP  maintains  the  use  of  ten  AAAV(P)  variants  and  one 
AAAV(C)  variant  for  DT-II,  maintains  the  type  of  tests  to  be  conducted  during  DT-II, 
and  maintains  the  twenty-three  month  DT-II  duration.  The  objectives  of  DT-II  are  listed 
as  [Ref.  57:p.  HI- 12]: 

•  Assess  AAAV  RAM. 

•  Assess  the  vehicle’s  ability  to  meet  all  technical  and  operational  performance 
requirements. 

•  Demonstrate  resolution  of  design  deficiencies  noted  in  earlier  testing. 

•  Demonstrate  readiness  for  production. 

•  Demonstrate  maturity  of  design  and  to  assess  vehicle’s  readiness  for 
operational  testing. 

•  Verify  that  technical  risks  have  been  minimized,  and  that  critical 
characteristics  have  been  achieved  (if  not  achieved,  then  technical  solutions 
have  been  identified). 

•  Support  Low  Rate  Initial  Production  (LRIP)  and  Full  Rate  Production  (FRP) 
decisions. 

•  Include  interoperability  between  P  and  C  variants  and  the  battlefield. 

•  Assess  ability  to  maneuver  with  other  mechanized  forces. 

•  Successfully  integrate  as  part  of  the  assault  force  in  OMFTS. 

•  Demonstrate  lethality,  and  survivability  requirements. 

During  DT-II,  a  second  Logistics  Demonstration  is  scheduled  to  demonstrate 
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improvements  made  from  the  lessons  learned  in  the  DT-I  Logistics  Demonstration.  The 
Logistics  Demonstration  is  scheduled  to  occur  prior  to  FUSL  and  IOT&E.  DT-II  will 
also  include  extensive  Reliability,  Availability,  and  Maintainability  -  Durability  (RAM- 
D)  operations  and  data  collection.  [Ref.  57:p.  III-l  1] 

Operational  testing  for  the  EMD  Phase  is  first  described  in  the  1988  Draft  TEMP 
as  serving  to,  “...provide  data  and  associated  analysis  on  the  operational  effectiveness 
and  suitability  of  the  AAAV  prior  to  Marine  Corps  Systems  Acquisition  Review  Council 
(MS ARC)  III.”  [Ref.  58:p.  23)  Acquisition  Plan  NAVSEA  88-043  and  Acquisition  Plan 
DRPM-AAA  90-002  give  a  similar  account,  but  would  use  the  OT-II  results  for  the 
Milestone-Ill  decision.  [Refs.  40,  79] 

The  1995  TEMP  lists  the  Initial  Operational  Test  and  Evaluation  (IOT&E)  as  the 
only  operational  test  to  occur  in  the  EMD  Phase.  IOT&E  would  be  a  three  part  test  using 
ten  to  twelve  AAAV(P)  vehicles  and  one  AAAV(C)  vehicle  to  demonstrate,  “the  ability 
to  achieve  all  required  operational  thresholds.”  [Ref.  59:p.  IV-9]  These  vehicles  would 
be  a  combination  of  four  LRIP  vehicles  and  nine  EMD  production  representative  vehicles 
that  were  refurbished  or  remanufactured  to  production  configuration,  and  therefore, 
IOT&E  could  be  used  to  support  the  Milestone  III  decision  and  the  decision  to  proceed 
into  full  rate  production.  [Ref.  59:p.  IV-9] 

Part  One  of  IOT&E  would  be  a  combined  DT/OT  test  of  the  AAAV’s 
functionality  and  mobility  in  both  tropical  and  arctic  environments.  Part  Two  would  be 
conducted  at  Camp  Pendleton,  and  would  feature  progressive  OTH  amphibious 
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operations  that  built  to  a  force-on-force  full  mission  profile  with  all  elements  of  a 
MAGTF.  Part  Three  of  the  IOT&E  would  be  conducted  at  29  Palms,  California  and 
would  feature  force-on-force  mechanized  offensive  and  defensive  operations.  [Ref.  59:pp. 
IV- 10  through  IV- 13] 

When  the  1995  TEMP  was  originally  submitted  to  DOT&E  in  1994,  the  TEMP 
was  returned  because  of  a  Milestone  date  shift.  The  cover  letter  from  DOT&E  and 
DTSE&E  on  the  returned  TEMP  addressed  MCOTEA’s  force  on  force  evaluation  plan  in 
the  TEMP  by  stating: 

The  methodology  for  the  conduct  and  evaluation  of  Initial  Operational 
Test  and  Evaluation  remains  unresolved.  Prior  to  Milestone  II,  the  Marine 
Corps  must  develop  a  scoring  system  that  provides  an  effective  method  of 
measuring  force  on  force  evaluations  with  real  time  casualty  assessment 
and  vehicle  removable  from  the  battlefield.  [Ref.  82] 

With  respect  to  LFT&E,  the  1995  TEMP  indicates  that  two  AAAV(P)  production 
representative  vehicles  would  undergo  twelve  months  of  LFT&E  at  the  Aberdeen  Proving 
Grounds  (APG)  based  on  APG’s  extensive  live  fire  testing  experience.  The  DRPM 
believes  that  because  the  AAAV(C)  variant  shares  the  same  hull  configuration,  additional 
LFT&E  for  the  AAAV(C)  would  be  unnecessary.  [Ref.  59:p.  III-17] 

As  with  DT,  the  1999  Draft  TEMP  makes  significant  changes  to  the  OT  plan. 
MCOTEA  adds  one  Operational  Assessment  (OA-1)  for  the  AAAV(C)  variant  in  the 
second  quarter  of  FY02,  another  Operational  Assessment  (OA-2)  in  the  first  half  of  FY04 
for  both  the  AAAV(C)  and  the  AAAV(P)  variant,  and  a  third  Operational  Assessment 
(OA-3)  in  the  third  quarter  of  FY04.  [Ref.  57:p.  II-8]  These  OAs  are  in  addition  to  the 
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already  scheduled  IOT&E,  which  now  is  to  occur  in  FY05. 

Operational  Assessment-1  will  serve  to  assess  the  AAAV(C)  variant’s  capabilities 
to  operate  in  an  amphibious  and  mechanized  environment  while  battalion  /  regimental 
commanders  and  staff  attempt  to  command  and  control  their  infantry  units  and  supporting 
arms.  The  TEMP  specifically  states,  “The  OA  will  focus  on  assessing  the  unique 
characteristics  of  the  AAAV  Command  Variant,  primarily  the  ability  of  the  battalion  or 
regimental  commander  to  command,  control,  and  communicate  with  senior,  subordinate, 
and  adjacent  units,  and  control  supporting  fires  during  OMFTS/STOM  operations.”  [Ref. 
57:p.  IV-11]  The  OA  will  compare  the  capabilities  of  the  AAAV(C)  with  the 
capabilities  of  the  AAVC7A1  while  focusing  on  the  AAAV(C)’s  following 
characteristics  [Ref.  57:p.  IV- 1 1]: 

•  Ability  of  the  Commander’s  staff  to  effectively  operate  the  workstations. 

•  Suitability  of  workstation  configuration  to  support  effective  staff  functioning, 
including  collaborative  planning  capability. 

•  Effectiveness  of  different  operational  environments  on  command  and  control 
capabilities. 

The  amphibious  assessment  will  occur  at  Camp  Pendleton  in  order  to  conduct  OTH  ship- 
to-shore  movements  while  the  mechanized  assessment  will  occur  in  29  Palms,  California 
in  order  to  conduct  larger  scale  MAGTF  operations. 

MCOTEA  identifies  limitations  to  the  OA  as  not  satisfying  all  performance 
characteristics  of  the  objective  system  because  the  vehicle  will  not  be  production 
representative.  The  vehicle  for  the  OA  will  be  the  same  converted  AAAV(P)  used  in 
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AAAV(C)  DT-I.  “Consequently,  any  conclusions  drawn  from  this  OA  will  be 
preliminary  and  will  be  subject  to  verification  during  testing  of  the  EMD  command 
variant  during  OA-3  and  IOT&E.”  [Ref.  57:p.  IV-13] 

OA-2  will  involve  the  assessment  of  three  AAAV(P)  production  representative 
vehicles  in  a  subarctic  cold  weather  environment.  Testing  will  include  combined  DT/OT 
special  events  and  OT  events  designed  to  assess  vehicle  design  improvements  since 
AAAV(P)  PDRR  prototype  testing.  [Ref.  57:p.  IV-13]  OA-2  has  the  following  specific 
objectives  [Ref.  57:p.  IV-14]: 

•  Assess  the  effect  of  below  freezing  air  temperatures  during  AAAV  high  water 
speed  and  transition  mode  operations,  specifically  icing  effects  as  a  result  of 
salt  spray  and  the  effects  on  mechanical  operation  of  above  /  below  waterline 
moving  parts,  intakes,  and  exhausts;  and  effects  on  waterborne  visibility. 

•  Assess  the  effects  of  subarctic  conditions  on  AAAV  land  mobility  and 
maneuverability,  visibility,  and  gunnery. 

•  Assess  the  ability  of  the  AAAV  Environmental  Control  System  to  cope  with 
subarctic  conditions,  and  maintain  operational  effectiveness  of  crew  and 
embarked  Marines. 

•  Assess  the  effect  of  subarctic  conditions  on  AAAV  signature. 

•  Assess  the  ability  of  other  AAAV  systems  and  subsystems,  displays,  and 
computer  hardware  and  software  to  cope  with  subarctic  conditions. 

•  Assess  the  human  factors  interface  associated  with  cold  weather  operations 
and  AAAV  operation,  controls  and  displays. 

•  Initial  assessment  of  AAAV  section  tactics,  techniques,  and  procedures. 

•  Assess  operational  and  troubleshooting  procedures  using  on-board  technical 
manuals  and  built-in  test  equipment. 
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•  Assess  logistical  and  maintenance  support  concepts  and  procedures. 

Afloat  tests  are  to  occur  in  Valdez,  Alaska  and  will  include  day,  night,  and 
twilight  water  operations  at  high  and  low  water  speeds.  Ashore  tests  are  to  occur  at  Ft. 
Greeley,  Alaska  and  will  include  day  and  night  land  mobility  and  gunnery  operations. 
MCOTEA  estimates  requiring  a  minimum  of  100  hours  of  land  operations  and  25  hours 
of  water  operations  for  meeting  OA-2  objectives.  [Ref.  57:p.  IV-14] 

OA-3  involves  the  assessment  of  one  AAAV(C)  and  five  AAAV(P)  production 
representative  vehicles  in  a  subtropical  environment.  The  vehicles  are  to  possess,  “all  of 
the  production  design  capabilities  and  like  hardware  for  form,  fit,  and  function.”  [Ref. 
57  :p.  IV- 16]  Testing  for  OA-3  will  also  include  some  combine  DT/OT  special  test  events 
for  gunnery,  signature,  and  the  high  humidity  environment.  Specific  test  objectives  for 
OA-3  include  [Ref.  57:p.  IV- 1 7] : 

•  Make  final  system  performance  assessment  of  both  AAAV  variants. 

•  Preliminary  assessments  of  AAAV  unit  contribution  to  armored/mechanized 
landing  forces. 

•  Rehearse  test  events  and  conditions  for  IOT&E. 

•  Continued  assessment  of  AAAV  section  tactics,  techniques,  and  procedures. 

•  Continued  assessment  of  operational  and  troubleshooting  procedures  using 
on-board  technical  manuals  and  built-in  test  equipment. 

•  Assess  logistical  and  maintenance  support  concepts  and  procedures. 

•  Assess  the  effects  of  subtropical  conditions  on  AAAV  water  and  land  mobility 
and  maneuverability,  visibility,  and  gunnery. 
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•  Assess  the  ability  of  the  AAAV  Environmental  Control  System  to  cope  with 
subtropical  conditions,  and  maintain  operational  effectiveness  of  crew  and 
embarked  Marines. 

•  Assess  AAAV  signature  under  representative  operational  conditions. 

•  Assess  the  ability  of  other  AAAV  systems  and  subsystems,  displays,  and 
computer  hardware  and  software  to  cope  with  subtropical  conditions. 

•  Assess  the  human  factors  interface  associated  with  high  heat  and  humidity 
operations  and  AAAV  operation,  controls  and  displays. 

•  Assess  the  suitability  of  the  AAAV(C)  staff  workstation  configuration  to 
support  effective  staff  functioning  and  collaborative  planning. 

•  Assess  the  placement  of  workstations  and  working  environment  in  the 
AAAV(C). 

•  Assess  the  ability  of  the  commander  to  communicate  with  senior,  subordinate, 
and  adjacent  units  and  control  supporting  fires  via  the  AAAV(C). 

OA-3  is  to  be  conducted  in  three  phases.  Phase  One  will  involve  day  and  night 
OTH  amphibious  operations  at  Camp  Pendleton  with  an  assault  amphibian  section 
operating  against  representative  threat  forces  ashore  and  afloat.  Phase  Two  is  a  training 
phase  in  preparation  for  the  next  phase.  Phase  Three  is  the  subtropical  environment 
assessment  scheduled  to  occur  at  Eglin  Air  Force  Base,  Florida.  Phase  Three  involves 
shore-to-shore  operations  designed  to  represent  OTH  movements,  and  ashore  operations 
having,  “inland  objectives  50-100  NM  [Nautical  Miles]  inland,  or  as  can  be  simulated  at 
Eglin  AFB.”  [Ref.  57:p.  IV-18]  The  force-on-force  events  will  be  designed  to  provide 
insight  into  data  collection  for  IOT&E  and  for  Force-on-Force  Real  Time  Casualty 


124 


Assessment  (FOF  RTCA).  MCOTEA  estimates  accumulating  a  minimum  of  500  hours 

of  land  operations  and  100  hours  of  water  operations.  [Ref.  57  :p.  IV- 19] 

The  1999  Draft  TEMP  increases  in  detail  its  description  of  IOT&E.  Specific 

changes  from  the  1995  TEMP  include  number  and  type  of  test  articles,  number  and 

scope  of  phases,  and  enhancements  to  FOF  RTCA. 

The  number  of  test  articles  for  IOT&E  has  decreased  from  the  originally 

planned  10-12  vehicles  to  eight  AAAV(P)s  and  one  AAAV(C).  These  vehicles  will 

equate  to  one  AAAV  platoon  (-)  and  an  AAAV(C)  command  element.  An  AAAV 

platoon(-)  represents  the  number  of  AAA  Vs  used  in  a  mechanized  infantry  company 

with  a  cross  attached  tank  platoon.  [Ref.  57:p.  IV-20]  This  mechanized  team,  or 

“Team  Mech,”  is  representative  of  existing  doctrinal  tactics.  The  IOT&E  test  articles 

are  to  be,  “production  representative  EMD  prototypes  that  contain  both  the  objective 

hardware  and  software  components  of  the  production  versions.”  [Ref.  57 :p.  IV-20] 

The  TEMP  states  the  following  IOT&E  objectives: 

The  IOT&E  results  will  support  the  MS-III  full  rate  production  decision. 

The  IOT&E  will  evaluate  the  operational  effectiveness  and  suitability  of 
the  AAAV  during  the  amphibious  assault;  AAAV(P)s  in  support  of 
platoon  and  company  mechanized  maneuver  elements;  and  the  AAAV(C) 
as  a  tactical  echelon  headquarters  for  an  infantry  battalion  or  regiment. 
Logistical  and  maintenance  support  concepts  will  be  fully  tested,  and 
RAM  data  will  be  collected  throughout  the  IOT&E.  The  IOT&E  will 
obtain  estimates  for  the  AoA's  [Analysis  of  Alternatives]  MOEs 
[Measures  of  Effectiveness]  with  the  Force  on  Force  (FOF)  Real  Time 
Casualty  Assessment  (RTCA),  and  provide  data  to  resolve  all  AAAV 
COIs...  [Ref.  57:p.  IV-20] 
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To  meet  these  objectives,  MCOTEA  plans  to  conduct  IOT&E  over  five  phases. 
Phase  One  involves  Pre-IOT&E  operator  and  maintainer  training  given  by  Assault 
Amphibian  School  personnel  at  Camp  Pendleton.  Phase  Two  will  be  Amphibious 
Operations  testing  at  Camp  Pendleton.  The  Amphibious  Operations  phase  is  designed  to, 
“evaluate  the  operational  effectiveness  and  suitability  of  the  AAAV  and  AAAV  units 
during  Over-The-Horizon  amphibious  operations,  and  to  resolve  the  Force  Build-up  and 
Movement  MOEs  [Measures  of  Effectiveness]  associated  with  the  ship-to-shore  phase  of 
landing  operations.”  [Ref.  57:p.  IV-21]  The  amphibious  phase  is  also  designed  to 
evaluate  the  AAAV  crews’  and  units’  ability  to  maintain  situational  awareness  and 
integrity  during  the  OTH  movement  to  shore  in  all  conditions  while  integrating  with 
LCACs  and  supporting  arms.  [Ref.  57:p.  IV-21] 

Phase  Three  involves  gunnery  operations  at  29  Palms,  California  on  an 
instrumented  live  fire  range.  Gunnery  operations  are  to  be  conducted  under  a  wide 
variety  of  different  conditions  based  on  established  AoA  scenarios.  The  phase  will  also 
evaluate  newly  established  gunnery  tables  designed  for  the  Mk  46  Weapon  Station.  [Ref. 
57:p.  IV-22] 

Phase  Four  of  IOT&E  evaluates  the  ability  of  the  AAAV  unit  to  participate  in 
well-established  Combined  Arms  Exercises  (CAX)  at  29  Palms,  California.  Combined 
Arms  Exercises  focus  on  a  regimental  headquarters’  ability  to  command  and  control  a 
battalion  that  is  conducting  live-fire  and  maneuver,  offensive  and  defensive  operations 
with  all  supporting  arms  available  to  the  MAGTF.  The  AAAV(P)  will  be  evaluated  for 
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its  ability  to  operate  in  a  training  exercise  in  support  of  a  mechanized  company  task  force. 
The  AAAV(C)  will  be  tested  and  evaluated  for  its  ability  to,  “perform  as  a  tactical 
echelon  headquarters,  provide  needed  connectivity  to  senior,  subordinate,  and  adjacent 
units,  and  to  coordinate  the  use  and  movement  of  designated  supporting  arms.”  [Ref. 
57:p.  IV-23] 

The  fifth  phase  of  IOT&E  addresses  the  concerns  expressed  by  DOT&E  and 
DTSE&E  in  their  comments  about  the  1994  TEMP  submission.  Force-on-Force  (FOF) 
field  testing  with  Real  Time  Casualty  Assessment  (RTCA)  is  to  be  conducted  for  a  30- 
day  period  immediately  following  the  CAX.  An  AAAV  equipped  force  will  be 
compared  to  an  AAV7A1  equipped  force  for  combat  effectiveness.  Specifically, 
“MCOTEA  will  use  the  most  current  FOF  RTCA  system  available  at  the  time  to 
provide  a  method  of  measuring  FOF  evolutions  with  RTCA  and  vehicle  removal  from 
the  battlefield.  Four  of  the  five  AoA  MOEs  can  be  resolved  via  FOF  field  testing  with 
RTCA.”  [Ref.  57:p.  IV-23] 

Scenarios  for  the  FOF  RTCA  will  be  developed  through  the  use  of  AoA 
modeling  and  simulation,  and  through  an  analysis  of  the  Operational  Mission 
Summary/Mission  Profile  (OMS/MP)  for  AAAV  capabilities  that  are  to  significantly 
enhance  the  supported  units.  [Ref.  57:p.  IV-23]  Scenario  scoring  of  the  four  evaluated 
MOEs  is  to  be  conducted  by  the  U.  S.  Army  Test  and  Experimentation  Command,  and 
is  listed  as  [Ref.  57 :p.  IV-23]: 

•  Loss-Exchange  Ratio.  Comparison  of  remaining  combat  power  of  AAAV 
equipped  blue  force  versus  remaining  combat  power  of  red  force  after  each 


127 


scripted  mission  and  remaining  combat  power  of  AAV  equipped  blue  force 
versus  remaining  combat  power  of  red  force  after  each  scripted  mission. 

•  Force  Movement.  Force  movement  will  be  scored  indirectly.  The  amount 
of  time  an  AAAV  equipped  blue  force  takes  to  accomplish  each  scripted 
mission  versus  amount  of  time  an  AAV  equipped  blue  force  takes  to 
accomplish  each  scripted  mission.  This  indirect  method  of  resolving  this 
issue  is  required  due  to  the  limited  amount  of  terrain  that  can  be 
instrumented. 

•  Fraction  of  Force  Surviving.  The  fraction  of  AAAV  equipped  blue  forces 
surviving  and  red  forces  surviving  after  each  scripted  mission  compared  to 
fraction  of  AAV  equipped  blue  force  surviving  and  red  force  surviving  after 
each  scripted  mission. 

•  Force  Ratio.  The  ratio  of  AAAV  equipped  blue  forces  and  red  forces 
surviving  after  each  scripted  mission  compared  to  the  ratio  of  AAV 
equipped  blue  forces  and  red  forces  surviving  after  each  scripted  mission. 


The  1999  Draft  TEMP  also  increases  it  descriptive  detail  concerning  LFT&E, 
with  emphasis  added  to  the  level  of  testing.  The  1995  TEMP’s  integrated  Test  Program 
Schedule  indicates  that  only  full-up  system  level  testing  would  occur  in  the  EMD  Phase. 
The  1999  Draft  TEMP  amplifies  the  DRPM’s  LFT&E  strategy  building  block  approach 
by  continuing  testing  throughout  EMD.  Ballistic  hull  and  turret  (BH&T)  testing  is  to 
continue  into  the  EMD  phase,  and  then  armor  and  component  ballistic  testing  and 
controlled  damage  testing  would  continue  until  full-up  system  level  (FUSL)  testing.  [Ref. 
57:p.  II-8]  The  testing  would  continue  use  of  extensive  modeling  and  simulation  to 
augment  testing  and  estimate  expected  vulnerability  performances.  [Ref.  57:p.  IV- 17] 
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5.  Production,  Fielding  /  Deployment,  Operations,  and  Sustainment  Phase 

The  DRPM’s  test  and  evaluation  strategy  for  the  PF/DOS  Phase  is  not  fully 
established  at  this  time.  The  references  to  production  testing,  FOT&E,  and  First  Article 
Testing  appear  to  be  merely  place  holders  in  all  versions  of  the  TEMP  until  future 
attention  can  be  applied  to  their  development. 

The  1988  Draft  TEMP  discusses  a  third  DT  event  (DT-III)  that  is,  “designed  to 
demonstrate  that  production  is  in  accordance  with  the  protection  data  package  and  that 
the  technical  and  engineering  requirements  have  been  satisfied  including  correction  of 
previously  identified  discrepancies.”  [Ref.  58:p.  18]  DT-III  was  to  be  conducted  by 
AVTB  with  detailed  test  plans  being  generated  by  the  government.  A  third  OT  event 
(OT-III)  would  test  production  vehicles  in  a  realistic  platoon  environment  with  the  first 
FMF  unit  equipped  with  the  AAAV.  OT-III  would  validate  the  correction  of  previously 
identified  deficiencies.  [Ref.  58:p.  24] 

The  1988  Draft  TEMP  does  dedicate  a  full  page  to  Production  Acceptance  Test 
and  Evaluation  (PAT&E).  It  identifies  the  purpose  of  the  acceptance  tests  as  ensuring  the 
production  items  meet  the  requirements  and  specifications  of  the  contract.  [Ref.  58:p.  25] 
Initial  production  tests  would  randomly  select  vehicles  in  the  first  three  months  of 
production  and  subject  them  to  600  hours  of  durability  testing.  Additionally,  100-hour 
control  and  comparison  testing  would  occur  on  production  vehicles  at  randomly  selected 
times.  [Ref.  58:p.  25] 
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The  1995  TEMP  states  that,  “Detailed  LRIP  and  Production  test  events,  scope, 

and  scenario  have  not  been  developed  for  the  AAAV  at  this  time.”  [Ref.  59:p.  III-12] 

However,  the  TEMP  does  identify  First  Article  Testing  (FAT)  for  two  LRIP  and  two 

production  vehicles  in  order  to  demonstrate  that  design  specifications  are  met,  and  that 

manufacturing  processes  are  acceptable.  [Ref.  59:p.  III-12]  The  TEMP  also  identifies 

possible  testing  of  another  ballistic  hull  and  turret  to  ensure  production  techniques  meet 

vulnerability  requirements.  [Ref.  59:p.  Ill- 18]  Part  Four  of  the  TEMP  addresses  Follow- 

on  Operational  Test  and  Evaluation  (FOT&E)  as  follows: 

The  AAAV  may  undergo  FOT&E  during  Production  and  Deployment  to 
evaluate  major  system  modifications  of  product  improvements,  along  with 
tactical  employment  within  the  MAGTF.  FOT&E  Test  events,  scope  of 
testing,  and  scenarios  will  depend  on  IOT&E  results  and  modifications 
made  after  IOT&E,  but  applied  before  full  rate  production.  FOT&E  will 
provide  operational  environments  for  evaluating  systems  against  any 
operational  deficiencies  or  shortcomings  found  during  IOT&E.  [Ref.  59:p. 

IV- 15] 

The  1999  Draft  TEMP  is  as  vague  about  production  testing  as  the  1995  TEMP. 
Verbiage  in  Part  Three  of  the  1999  Draft  TEMP  mirrors  its  predecessor  with  the 
exception  that  only  two  LRIP  and  no  production  vehicles  would  be  used  for  FAT.  [Ref. 
57:pp.  Ill- 14,  III- 15]  The  1999  Draft  TEMP’s  Part  Four  references  to  FOT&E  are 
verbatim  to  that  listed  above  for  the  1995  TEMP.  [Ref.  57:p.  IV-26] 


E.  REQUIREMENTS  DEVELOPMENT  AND  RECONCILIATION  PROCESS 

The  goal  of  the  Program  Manager  is  to  field  a  system  that  meets  the  needs  and 
requirements  of  the  user.  Thus,  the  development  and  evolution  of  requirements  serve  as  a 
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reference  point  from  which  the  program’s  test  and  evaluation  strategy  is  based. 
Reconciliation  of  system  requirements  throughout  the  system’s  development  ensures  that 
the  final  product  can  actually  perform  as  desired. 

1.  Initial  Requirements 

Prior  to  entering  the  Milestone  0  process,  the  AAAV  program  office  focused  on 
meeting  the  fundamental  technical  requirements  for  developing  a  high  water  speed 
assault  amphibian  vehicle.  The  1988  Draft  TEMP  lists  the  following  Required  Technical 
Characteristics  deemed  necessary  for  the  AAAV  [Ref.  58:p.  7]: 

•  Tracked  Vehicle. 

•  Must  carry  a  reinforced  Squad  (17-18  Marines). 

•  Ml  A1  Tank  /  Future  Main  Battle  Tank  (MBT)  mobility. 

•  14.5  millimeter  (mm)  hull  protection  without  applique  armor. 

•  Weapon  system  to  defeat  Soviet  BMP  (armored  personnel  carrier)  threat. 

•  Support  dismounted  infantry. 

•  Night  /  All  weather  capability. 

•  Accurate  fire  while  moving  on  both  land  and  water. 

•  High  water  speed  of 20-25  knots  in  Sea  State  Three  (SS-3). 

•  Slow  water  speed  of  at  least  10-12  knots. 

•  Survive  in  Sea  State  Five  (SS-5). 

•  Negotiate  8-10  Foot  plunging  surf. 
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•  Self-navigation  at  sea. 

•  Maximum  crew  of  three  (Driver,  Commander/Gunner,  Assistant  Driver). 

•  Maximum  commonality  with  other  systems  (U.  S.  Army  Armored  Family  of 
Vehicles). 

The  technical  annex  to  Acquisition  Plan  NAVSEA  88-043  Revision  1  (89) 
identifies  very  similar  characteristics  as  constraints  to  the  AAAV's  acquisition.  The 
acquisition  plan  states  in  broad  terms  that,  "The  AAAV  must  be  compatible  with 
approved  U.S.  Marine  Corps  and  U.S.  Navy  amphibious  doctrine,  tactics,  and 
equipment."  [Ref.  40:p.  9]  In  more  specific  terms,  the  acquisition  plan  adds  that  the 
AAAV  must  be  capable  of  high  water  speeds  greater  than  20  miles  per  hour,  and  that  it 
will  be  a  fully  amphibious  armored  personnel  carrier  possessing  a  weapon  system 
sufficient  to  [Ref.  40  :p.  9]: 

•  Provide  effective  direct  offensive  and  defensive  fire  support  for  mounted  and 
dismounted  infantry. 

•  Provide  sufficient  firepower  to  destroy  the  threat  BMP  of  the  timeframe. 

Acquisition  Plan  DRPM-AAA  90-002  makes  only  slight  changes  to  the  technical 

constraints  listed  in  NAVSEA  88-043.  The  1990  acquisition  plan  specifies  that  the 
AAAV  achieve  high  water  speeds  of  grater  than  20  knots  vice  miles  per  hour,  and  that  the 
AAAV  be  capable  of  transporting  a  reinforced  rifle  squad  and  "associated  equipment". 
[Ref.  79:p.  10] 
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2.  Cost  and  Operational  Effectiveness  Analysis  (COEA) 

During  the  Milestone  0  decision  process,  the  PMO  was  directed  to,  “examine  all 
alternatives  of  placing  infantry  ashore,  not  just  a  new  amphibious  vehicle.”  [Ref.  60:p.  1] 
Based  on  this  guidance,  the  PMO  expanded  the  Cost  and  Operational  Effectiveness 
Analysis  (COEA)  from  three  systems  to  thirteen  possible  systems.  The  thirteen  systems 
fell  into  four  general  categories.  The  first  category  of  “high-speed  amphibians”  contained 
the  AAAV(Fast)  which  would  be  a  newly  designed  system,  and  the  AAV7A2  which 
would  be  an  improved  AAV7A1  capable  of  achieving  high-water  speed.  [Ref.  60:p.  1] 

The  second  category  was  designated  “low-speed  amphibians”  and  contained  four 
possible  systems.  The  existing  AAV7A1  would  serve  as  a  baseline  system.  The 
AAV7A2(Slow)  would  contain  product  improvements  to  the  existing  system,  but  would 
not  attain  high  water  speed.  The  AAAV(Slow)  would  be  a  newly  designed  vehicle 
capable  of  only  low  water  speeds.  The  fourth  low-speed  candidate  was  a  fully 
submersible  tracked  vehicle  that  would  transit  to  the  shore  below  the  water’s  surface. 
[Ref.  60:p.  1] 

The  third  category  of  possible  systems  was  designated  “non-amphibian”  and 
contained  five  possible  systems.  Receiving  consideration  was  the  Armored  Personnel 
Carrier  (Experimental)  (APC(X))  which  would  be  a  newly  designed  APC.  Other  “non¬ 
amphibians”  receiving  consideration  were  the  USMC’s  current  LAV-25  wheeled  vehicle, 
the  U.S.  Army’s  M2A2  Bradley  Fighting  Vehicle,  the  U.S.  Army’s  M113A3  APC,  and 
the  U.S.  Army’s  Future  Infantry  Fighting  Vehicle  (FIFV).  [Ref.  60:p.  1] 
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The  final  category  of  possible  systems  was  designated  “non-vehicle”  and 
contained  only  two  options.  The  surface  option  examined  bringing  infantry  ashore  in 
shelters  mounted  on  LCACs.  The  air  option  was  to  simply  bring  all  infantry  ashore  with 
an  existing  helicopter  system.  For  the  COEA,  the  U.S.  Army’s  CH-60  was  used  as  the 
representative  aircraft.  [Ref.  60:p.  5] 

The  original  COEA  began  by  analyzing  the  possible  systems  in  the  areas  of 

system  performance,  force  effectiveness,  and  life-cycle  cost.  Six  systems  were 

eliminated  based  on  performance  while  the  remaining  systems  were  analyzed  for  their 

contribution  to  force  capabilities.  The  systems  were  compared  in  force-on-force 

scenarios  using  established  force-level  Measures  of  Effectiveness  (MOEs)  resulting  in 

three  more  systems  being  screened  out.  [Ref.  60  :p.  1 1]  The  remaining  systems  were  then 

compared  for  life-cycle  costs.  As  a  result,  the  AAAV(Fast),  or  AAAV,  was  determined 

to  best  meet  the  needs  of  the  Marine  Corps.  [Ref.  60:p.  11] 

In  1993,  the  COEA  was  updated  for  the  next  milestone  review.  That  COEA 

examined  four  new  alternatives,  which  examined  different  combinations  of  high-speed 

and  low-speed  vehicle  employment.  Again,  the  AAAV  was  determined  to  be  the  best 

system  for  the  Marine  Corps  with  the  following  justification: 

The  AAAV(F)  is  the  most  promising  and  operationally  effective 
evaluated.  Its  inherent  characteristics  provide  an  amphibious  force  with 
the  ability  to  use  speed  as  a  weapon;  through  the  rapid  concentration  of  its 
available  combat  power  at  a  decisive  place  and  time.  All  other 
alternatives. .  .inhibit  our  ability  to  seize  the  initiative  and  dictate  the  terms 
of  combat  by  forcing  the  enemy  to  react  to  us.  Without  this  initiative  an 
amphibious  force’s  survivability  and  security  is  greatly  diminished,  along 


134 


with  the  loss  of  essential  elements  that  enable  maneuver  and  surprise. 

[Ref.  60:p.  14] 

The  DRPM  further  justified  the  selection  of  the  AAAV  at  the  1994  Acquisition 
Review  Board  by  stating,  “Alternatives  other  than  the  AAAV(F)  tend  to  move  the 
amphibious  force  in  a  direction  of  an  attrition  style  warfare  instead  of  in  the  direction  of 
maneuver  warfare,  which  is  the  doctrinal  foundation  for  all  U.  S.  Marine  Corps 
operations.”  [Ref.  67] 

An  extract  from  the  Integrated  Program  Summary  is  provided  in  Appendix  G. 
The  extract  covers  an  extensive  review  of  the  thirteen  candidate  systems,  and  the 
justification  for  selection  or  non-selection  of  each  system  or  mix  of  systems. 
Additionally,  Appendix  H  provides  a  description  of  the  selected  AAAV  system  as  it 
appears  in  the  1999  Draft  TEMP. 

3.  Key  Performance  Parameters 

Based  on  the  results  of  the  initial  COEA  of  1991,  the  DRPM  sought  approval 
from  the  Joint  Requirements  Oversight  Council  (JROC)  for  the  objective  and  threshold 
values  of  the  required  capabilities  shown  in  Table  4  [Ref.  83]: 
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Performance 

Objective  /  Threshold 

High  Water  Speed  (Sea  State-Three) 

25  knots  /  20  knots 

Transition  Mode 

8.6  knots  /  7  knots 

Surf  Capability 

10  feet  /  8  feet 

Cross  Country  Speed 

30  mph  /  30  mph 

Armor  Protection 

30mm@  1000m  /  14.5mm@  300m 
Frontal  60°  Arc  /  any  azimuth 

Firepower 

BMP-3(+30mm)  /  BMP-3(+13mm) 

Troop  Capacity 

18/18 

Mean  Time  Between  Critical  Mission 
Failure  (MTBCMF) 

58  hrs  /  39  hrs 

Table  4.  1991  AAAV  Required  Capabilities.  [Ref.  83] 


The  establishment  of  these  values  was  justified  to  the  JROC  in  terms  of 
operational  significance.  Threshold  high  water  speed  was  the  minimum  necessary  to  stay 
within  acceptable  ride  quality  criteria  established  by  Fatigue  Decreased  Proficiency  Mil- 
Std-1472B.  Objective  speed  was  established  to  provide  a  margin  accommodating  range 
and  sea  state  error,  and  additional  hydrography  standoff  while  still  maintaining  Mil-Std 
criteria.  [Ref:  83] 

Transition  mode  threshold  speed  was  the  minimum  speed  necessary  for  safe 
transit  of  the  surf  zone  to  prevent  broaching  of  the  vehicle.  Objective  transition  mode 
speed  provided  a  safety  margin  for  surf  zone  transit.  Surf  capability  threshold  and 
objective  heights  were  based  on  a  performance  band  beyond  which  joint  operations  with 
landing  craft  became  no  longer  practical.  Cross  country  speed  threshold  and  objective 
values  provide  sufficient  capability  to  permit  employment  in  battlefield  formations  with 
the  Main  Battle  Tank  of  the  time  frame.  [Ref:  83] 
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Armor  protection  threshold  values  are  based  on  the  most  likely  or  most  abundant 

threat  light  armored  vehicle  gun  caliber,  while  objective  values  are  based  on  providing 

protection  against  the  remaining  threat  light  armor  vehicles  expected  to  be  encountered 

and  engaged.  Firepower  threshold  values  are  based  on  estimated  armor  levels  plus  the 

minimum  established  behind-armor  effects  encountered  at  minimum  required 

engagement  ranges.  The  objective  values  are  based  on  penetration  capabilities  plus  the 

behind-armor  effects  at  desired  engagement  ranges.  [Ref:  83] 

Troop  carrying  capability  threshold  and  objective  values  are  based  on  retaining 

the  integrity  of  the  smallest  tactical  unit  (reinforced  rifle  squad)  that  prevents  increases  in 

manpower  requirements,  and  that  retains  doctrinal  lift  requirements  within  shipping 

constraints.  Reliability  threshold  and  objective  values  would  provide  an  88%  and  92% 

respective  probability  of  successfully  completing  the  primary  mission.  [Ref:  83] 

In  response  to  the  DRPM’s  request,  the  JROC  made  three  specific 

recommendations  to  the  Marine  Corps.  First,  "That  the  high-water-speed  performance 

objective  be  stated  in  terms  of  speed  over  a  given  time,  i.e.  endurance."  Second,  that  the 

Mean  Time  Between  Critical  Mission  Failure  (MTBCMF)  performance  objective  "be 

stated  as  a  range,  vice  a  specific  number."  Third,  that  the  performance  objectives  convey 

the  requirement  for  "continuous  mobility  of  Marine  forces."  [Ref.  84]  In  the  Draft 

Memorandum  for  the  Under  Secretary  of  Defense  for  Acquisition,  the  Vice  Chairman  of 

the  Joint  Chiefs  of  Staff  concludes  that: 

...the  following  are  key  thresholds/goals  for  the  acquisition  program 
baseline:  (a)  attain  a  water  speed  of  20-25  knots  in  sea  state  three  for  not 
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less  than  one  hour  duration;  (b)  provide  continuous  overland  mobility  at  a 
speed  which  will  permit  tactical  employment  with  the  main  battle  tank  in 
use  during  the  AAA V's  projected  service  life;  (c)  provide  improved  armor 
protection  over  the  existing  system;  (d)  provide  the  carrying  capacity  for  a 
reinforced  Marine  Rifle  Squad  (eighteen  combat  equipped  Marines);  (e) 
possess  sufficient  firepower  to  defeat  threat  light  armored  vehicles;  (f)  be 
sufficiently  reliable  to  accomplish  the  critical  elements  of  the  mission 
profile  and  still  achieve  a  unit  C-l  readiness  status.  [Ref.  84:Tab  1] 

Table  5  identifies  the  Key  Performance  Parameters  (KPPs)  as  revalidated  by  the  JROC  on 

16  February  1995,  and  reported  in  DRPM  AAA  Selected  Acquisition  Reports. 


Performance  Characteristics 

Objective  /  Threshold 

High  Water  Speed  (SS-3,  36  inch 
Significant  Wave  Height) 

25  knots  /  20  knots 

Forward  Speed  on  a  Hard  Surface  Road 

72  kph  /  69  kph 

Armor  Protection  Against 

30mm@1000m  /  14.5mm@300m 

Carry  Capacity  (Marines) 

18/17 

Firepower  (Maximum  Effective  Range) 

2000  meters  /  1500  meters 

Reliability  -  Mean  Time  Between  Critical 
Mission  Failure  (MTBCMF) 

95  hrs  /  70  hrs 

_ 

Table  5.  1995  JROC  Approved  Key  Performance  Parameters  for  AAAV.  [Ref.  71] 


Based  on  the  Key  Performance  Parameters,  the  following  Milestone  II  exit  criteria 
were  developed  and  approved  [Ref.  57:p.  III-4]: 

•  Complete  1000  hour  engine  durability  test. 

•  Demonstrate  speed  of  20  knots  in  Sea  State  2  for  1  hour. 

•  Demonstrate  64  kph  forward  speed  on  a  hard  surface  road. 

•  Validate  armor  performance: 

-  95%  Probability,  no  penetration  from  14.5mm  @  300m. 

-  99%  Probability,  no  penetration  from  1 55mm  fragments  airburst  @50 
feet  above  the  top  of  the  vehicle. 
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4.  Critical  Technical  Parameters 

Once  the  COEA  confirmed  the  AAAV  as  the  desired  system,  MCCDC  developed 
and  approved  the  Operational  Requirements  Document  (ORD)  in  1994.  After  outlining 
the  AAV7Al's  deficiencies,  the  ORD  describes  the  AAAV's  required  capabilities  in  the 
context  of  Operational  Maneuver  From  the  Sea  (OMFTS)  /  Over  the  Horizon  (OTH) 
doctrine,  and  with  respect  to  the  anticipated  threat  capabilities  of  the  future.  [Ref.  2]  In 
broad  terms  the  ORD  states,  "The  threat  to  naval  amphibious  forces  in  the  21st  century 
will  encompass  the  entire  operational  spectrum  of  military  capabilities  ranging  from 
dissident  /  guerrilla  forces  to  sophisticated  first  line  equipped,  regular  forces."  [Ref.  2:p. 
8]  The  ORD's  system  performance  overview  specifically  states  that,  "The  AAAV  must 
have  the  water  mobility  to  support  FTS  [From  the  Sea]  and  OMFTS  concepts  of 
operations  and  be  compatible  with  naval  amphibious  shipping  of  the  2005-2025  time 
frame... The  AAAV  will  be  subject  to  worldwide  employment  in  all  conflict 
environments."  [Ref.  2:p.  12] 

The  ORD  goes  on  to  describe  the  detailed  operational  requirements  that  the 
AAAV  must  be  capable  of  performing.  The  ORD,  the  Acquisition  Program  Baseline,  and 
the  1995  TEMP  all  list  the  same  six  critical  system  characteristics.  These  six 
characteristics  match  the  KPPs  established  by  the  JROC  in  1995  (Table  5),  and  serve  as  a 
foundation  for  the  AAAV  program.  [Ref.  59:p.  1-6] 

The  system's  Critical  Technical  Parameters  (CTPs),  listed  in  both  the  1995  TEMP 
and  the  1999  Draft  TEMP,  are  a  derivative  of  the  extensive  ORD  requirements.  The 
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1999  Draft  TEMP  lists  the  following  major  categories  of  CTPs,  and  number  of  CTPs  per 
category  [Ref.  57:p.  1-14]: 

•  Survivability  (7). 

•  Land  Mobility  (12). 

•  Water  Mobility  (8). 

•  Reliability,  Availability,  Maintainability  (7). 

•  Interoperability  (5). 

•  Logistics  /  Supportability  /  Human  Factors  (10). 

•  Firepower  -  AAAV(P)  (4). 

•  AAAV(C)  Unique  (4). 

Appendix  I  lists  all  AAAV  Critical  Technical  Parameters  as  defined  in  the  1999  Draft 
TEMP. 


5.  Critical  Operational  Issues 

In  Part  IV  of  the  1995  TEMP,  MCOTEA  describes  their  approach  to  the  AAAV’s 
evaluation  as  follows: 

The  purpose  of  operational  test  and  evaluation  is  to  provide  information  to 
support  decisions  that  consider  the  operational  effectiveness  and  suitability 
of  a  system.  This  is  regularly  accomplished  by  resolving  Critical 
Operational  Issues  (COIs)  or  questions  that  are  considered  operationally 
significant.  Operational  test  and  evaluation  provides  a  basis  for  the 
resolution  of  COIs  by  comparing  system  characteristics  to  operational 
requirements  in  a  realistic  operating  environment... [FJavorable  resolution 
of  every  COI  will  result  in  the  system  being  operationally  effective  and 
suitable.  [Ref.  59:p.  IV-1] 
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MCOTEA  establishes  and  lists  three  COIs  for  evaluation  of  the  AAAV’s  operational 
effectiveness,  five  COIs  for  the  AAAV’s  operational  suitability,  three  AAAV(P)  unique 
COIs,  and  two  AAAV(C)  unique  COIs.  The  COIs  listed  in  the  1995  TEMP  are  as 
follows  [Ref.  59:pp.  IV-2,3]: 

Operational  Effectiveness  COIs 

•  Does  the  AAAV  support  the  Marine  Corps’  concept  for  Over-The-Horizon 
(OTH)  amphibious  assault  and  other  amphibious  operations  worldwide? 

•  Is  the  AAAV  capable  of  conducting  sustained  operations  over  varied  terrain, 
at  speeds  permitting  tactical  employment  with  the  Main  Battle  Tank  (MBT) 
and  other  combat  vehicles? 

•  Does  the  AAAV  meet  the  Marine  Corps’  requirement  for  survivabilty? 
Operational  Suitability  COIs 

•  Does  the  AAAV  meet  Marine  Corps  requirements  for  reliability,  availability, 
and  maintainability  (RAM)? 

•  Is  the  AAAV  transportable  by  means  of  land,  sea,  and  air  transport? 

•  Can  the  AAAV  be  operated  and  maintained  in  accordance  with  the  Marine 
Corps’  plans  for  personnel  selection,  training,  organization,  concept  of 
employment  and  logistic  support? 

•  Does  the  design  of  the  AAAV  provide  for  Marine-machine  interface,  and  safe 
operations  and  maintenance? 

•  Is  the  AAAV’s  software  usable  and  maintainable? 

AAAV(P)  Variant  Unique  COIs 

•  Will  a  reinforced  combat-equipped  Marine  rifle  squad  be  mission-effective 
after  transportation  over  land  and  sea  in  a  AAAV(P)? 
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•  Is  the  AAAV(P)  armament  effective,  including,  lethality,  against  the  lightly 
armored  threat  targets  specified  in  the  System  Threat  Analysis  Report 
(STAR)? 

•  Does  the  AAAV(P)  enable  the  embarked  Marine  rifle  company  or  platoon 
commander  to  command  and  control  subordinate  units,  communicate  with 
higher  and  adjacent  units,  and  coordinate  supporting  arms  with  supporting  and 
supported  units  during  amphibious  assaults  and  operations  ashore? 

AAAV(C)  Variant  Unique  COIs 

•  Can  the  AAAV(C)  transport  the  Marine  infantry  regimental  or  battalion 
commander  and  required  staff? 

•  Does  the  AAAV(C)  enable  the  embarked  regimental  or  battalion  commander 
and  staff  to  command  and  control  subordinate  units,  communicate  with 
higher,  lower,  and  adjacent  units,  and  coordinate  supporting  arms  with 
supported  and  supporting  units? 


MCOTEA  makes  significant  changes  to  the  description  of  their  evaluation 

approach  and  to  the  COIs  listed  in  the  1999  Draft  TEMP.  A  more  detailed  summary  of 

MCOTEA’s  evaluation  approach  to  OT&E  includes,  “Use  of  test  scenarios  that  focus  on 

AAAV  resolution  of  Critical  Operational  Issues,  and  integration  into  future  fighting 

concepts  to  aid  in  validation  of  AAAV  tactics.”  [Ref.  57:p.  IV- 1]  MCOTEA’s 

description  of  critical  operational  issues  is  expanded  as  follows: 

The  AAAV  COIs  are  directly  traceable  to  the  performance  of  the  AAAV 
mission  in  intended  warfighting  concepts  such  as  Operational  Maneuver 
From  the  Sea/Ship-To-Objective-Maneuver  (OMFTS/STOM),  and  during 
Sustained  Operations  Ashore  (SOA)... Satisfaction  of  the  COIs  over  the 
course  of  the  AAAV  OT&E  Program,  and  subsequent  evaluation  of 
system  operational  effectiveness  and  suitability  will  significantly  reduce 
program  risk  and  provide  required  information  on  system  performance  and 
force  level  effectiveness  to  senior  decision-makers.  [Ref.  57:p.  IV-3] 
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The  newly  developed  COIs  are  designed  to,  “capture  the  top  level  issues  associated  with 
AAAV  system  and  mission  level  performance”  and  are  as  follows  [Ref.  57:p.  IV-3]: 
Operational  Effectiveness  COIs 

•  Does  an  AAAV-equipped  amphibious  force  support  the  Navy-Marine  Corps 
Team's  concept  of  an  Over-The-Horizon  (OTH)  amphibious  assault 
capability? 

•  Does  an  AAAV-equipped  mechanized  force  significantly  enhance  the  force’s 
ability  to  fight  and  survive  during  combat  operations  ashore  against  future 
threat  forces? 

Operational  Suitability  COI 

•  Is  an  AAAV-equipped  force  reliable  and  supportable  throughout  anticipated 
wartime  and  peacetime  operations? 

Part  IV  of  the  1999  Draft  TEMP  further  states  that  these  new  COIs  are  directly  linked  to: 
the  Mission  Needs  Statement;  the  Acquisition  Program  Baseline  Key  Performance 
Parameters  and  the  Operational  Requirements  Document;  and  the  Measures  of 
Effectiveness  discussed  in  the  Analysis  of  Altematives/COEA.  [Ref.  57:p.  IV-3] 

6.  Measures  of  Effectiveness  and  Measures  of  Performance 

Measures  of  Effectiveness  (MOEs)  and  Measures  of  Performance  (MOPs)  for  the 
AAAV  were  developed  to  support  the  initial  COEA.  The  COEA  analyzed  the  candidate 
systems  from  a  system  performance  perspective.  The  MOPs  developed  to  analyze 
performance  include  ship-to-shore  movement,  mobility  ashore,  survivability  and  lethality. 
[Ref.  60  :p.  11]  These  performance  measures  were  used  to,  “screen  the  least  capable 
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systems  from  the  group  in  order  to  concentrate  the  bulk  of  analytical  resources  on  the 
more  promising  alternatives.”  [Ref.  60:p.  1 1] 

The  COEA  then  used  MOEs  at  the  force  level  in  order  to  compare  the  remaining 
systems  in  force-on-force  scenarios.  The  force-on-force  scenarios  covered  the  full  range 
of  combat  intensity  levels  and  analyzed  the  different  candidate  systems’  contribution  to 
the  force’s  overall  capability  to  conduct  amphibious  operations.  The  MOEs  developed 
for  COEA  effectiveness  analysis  include  Loss  Exchange  Ratio  (LER),  force  movement, 
percentage  of  force  survival,  force  ratio,  and  time  to  move  the  assault  element  from  ship 
to  shore.  [Ref.  60  :p.  11]  Table  6  is  from  the  1995  TEMP  and  shows  the  relationship 
between  the  AAAV  MOEs  and  MOPs,  and  establishes  the  basis  for  testable  criteria  from 
the  ORD  requirements. 

7.  Specifications  and  Detailed  Test  Plans 

General  Dynamics  has  developed  both  the  specification  and  the  detailed  test  plan 
for  DT-I  with  governmental  input  and  approval.  The  specification  for  the  AAAV(P) 
serves  as  the  Functional  Baseline  for  the  AAAV  Program  and  is  titled,  Advanced 
Amphibious  Assault  Vehicle  Personnel  Vehicle  System/Subsystem  Specification.  [Ref. 
85]  The  specification  identifies  the  technical  and  mission  requirements  for  the  AAAV  as 
an  entire  system,  allocates  requirements  to  functional  areas,  documents  design 
constraints,  and  defines  the  interfaces  between  and/or  among  the  functional  areas.  The 
document  further  states,  “This  specification  is  based  upon  the  Operational  Requirements 
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Document  (ORD)  and  the  technical  /  functional  parameters  developed  during  the  Concept 
Exploration  and  Definition  Phase  of  the  program.”  [Ref.  85] 
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Table  6.  AAAV  MOE  and  MOP  testable  criteria  relationships.  [Ref.  59:p.  1-8] 


The  specification  establishes  the  precedence  of  various  requirements 
documentation  as  follows: 

In  the  event  of  conflict  between  the  ORD  and  the  contents  of  this 
specification,  the  contents  of  this  specification  shall  be  considered  as  the 
superseding  requirement.  Where  differences  between  the  ORD  and  the 
Mission  Essential  Functions/Characteristics  (as  stated  in  the  Wartime 
Mission  Profile/Operational  Mode  Summary  and  Peacetime  Mission 
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Profile)  exist,  the  contents  of  the  ORD  shall  be  considered  as  the 
superseding  requirement.  [Ref.  85] 

The  combine  DRPM-AAA  and  GDAMS  test  plan  for  DT-I  was  established  in  1 
June  1999  as  Contract  Data  Requirements  List  (CDRL)  T005  under  the  AAAV 
Demonstration  and  Validation  Contract  of  13  June  1996.  [Ref.  86]  The  test  plan  lists  its 
objective  as,  “...to  validate  at  the  system/vehicle  level  all  of  the  performance 
requirements  for  the  Milestone  II  exit  criteria;  the  Critical  Technical  Parameters  as 
defined  in  the  TEMP;  and  the  test  parameters  of  the  AAAV  System  Validation  Matrix 
contained  in  the  System/Segment  Specification.”  [Ref.  86]  The  test  plan  is  actually 
divided  into  five  subtest  plans,  or  “Books,”  in  order  to  facilitate  use  and  transmittal.  The 
books  are  Land  Mobility,  Water  Mobility,  Firepower,  Specialty,  and  Electromagnetic 
Compatibility/Electromagnetic  Interference  (EMC/EMI).  [Ref.  86] 

The  test  plan  identifies  the  test  process  as  first  focusing  on  vehicle  operational 
safety  and  proper  system  functionality.  The  process  would  then  be  conducted  in  a  spiral 
nature  such  that  performance  and  functional  tests  are  gradually  increased  in  complexity 
and  risk.  [Ref.  86 :p.  6]  The  objectives  for  land  and  water  mobility  testing  include 
validation  of  the  land  and  water  mobility  performance  requirements  listed  in  the  AAAV 
System  Validation  Matrix  of  the  System/Segment  Specification.  [Ref.  86]  The  Firepower 
testing  objective  is  identified  as,  “...to  provide  confidence  that  the  weapon  station  design 
meets  the  requirements  defined  in  the  SOW  [Statement  of  Work]  regarding  performance 
and  maintainability.”  [Ref.  86]  The  objectives  for  Signature  testing  are  designed  to 
collect  data  that  allows  “as-built  signature  performance”  to  be  compared  with  thermal 
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infrared,  radar,  acoustic,  and  magnetic  signature  performance  requirements.  [Ref.  86] 
Similar  objectives,  “...  to  verify  that  the  AAAV  meets  all  requirements  specified  in  the 
AAAV  system  specification...,”  are  listed  for  EMC  and  EMI  testing.  [Ref.  86] 

8.  Requirements  Reconciliation 

The  purpose  of  requirements  reconciliation  is  to  ensure  that  the  desired 
performance  characteristics  sought  by  the  user  match  those  characteristics  achieved  by 
the  production  system.  The  reconciliation  of  requirements  generally  occurs  at  three 
levels.  First,  the  requirements  listed  in  the  ORD  are  reviewed  and  updated  for  each 
Milestone.  MCCDC  is  currently  reviewing  and  revising  the  AAAV  ORD  in  preparation 
for  Milestone  II  using  an  IPT  process.  The  ORD  IPT  is  chaired  by  the  AAAV  ORD 
Action  Officer  from  MCCDC  and  contains  members  from  MCOTEA,  DRPM  AAA, 
DOT&E,  and  other  as  required.  [Ref.  87] 

On  6  April  1999,  the  DRPM  made  a  request  to  MCCDC  for  48  clarifications  to 
the  ORD  in  preparation  for  the  first  EOA.  The  requested  clarifications  generally  seek 
more  detailed  definitions  or  more  specific  performance  criteria.  [Ref.  88]  The  DRPM’s 
request  specifically  states,  “Extensive  planning  to  execute  the  EOA  is  currently  ongoing. 
A  critical  step  in  operational  test  planning  is  ensuring  the  relationship  between  the  vehicle 
design  and  operational  requirements  are  explicitly  articulated  for  effective  assessment  of 
vehicle  operational  performance.”  [Ref.  88] 

Based  on  the  DRPM’s  original  request,  MCOTEA  submitted  an  ORD  clarification 
request  that  sought  even  greater  detail  from  MCCDC.  On  22  and  23  July  1999, 
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MCOTEA  held  an  ORD  clarification  meeting  in  order  to  assist  in  the  reconciliation 
process.  Attendees  from  MCCDC,  DRPM  AAA,  and  MCOTEA  worked  through 
MCOTEA’ s  concerns  and  identified  acceptable  verbiage  that  would  be  included  in  the 
updated  ORD.  [Ref.  89]  MCOTEA’s  concerns  involved  requirements  that  were  difficult 
to  measure  in  an  operational  environment,  and  terminology  that  was  inconsistent  through 
out  the  ORD  and  conflicted  with  what  was  contained  in  the  specification.  Of  note,  the 
representative  from  each  of  the  attending  agencies  was  an  experienced  officer  from  the 
Assault  Amphibian  Vehicle  community.  [Ref.  89] 

The  second  level  of  requirements  reconciliation  occurs  during  the  update  of  the 
TEMP.  The  TEMP  ensures  that  specified  ORD  requirements  are  translated  into  Critical 
Technical  Parameters,  and  that  the  established  COIs  can  be  met  by  the  established  MOEs 
and  MOPs.  The  1999  Draft  TEMP’s  listing  of  CTPs  references  specific  ORD  paragraphs 
to  ensure  continuity.  [Ref.  57:p.  1-15]  Continuity  for  COIs  is  addressed  in  Part  IV  of  the 
1999  Draft  TEMP  as  follows: 

MCOTEA  will  review  each  update  of  the  AAAV  Program  Analysis  of 
Alternatives  (AoA,  formerly  the  COEA)  and  update  the  TEMP  to  ensure 
linkage  among  COEA/AoA  MOEs,  the  TEMP  COIs/  MOEs,  and  the  ORD 
thresholds.  [Ref.  57 :p.  IV-3] 

Finally,  the  contractor  and  the  DRPM  ensure  that  established  requirements  are 
translated  into  specifications,  and  that  the  system,  subsystems,  and  components  are  tested 
to  meet  those  specifications.  General  Dynamics  conducted  initial  requirements  analysis 
from  13  June  1996  when  the  PDRR  phase  contract  was  awarded  to  16  April  1997  when 
the  System  Design  Review  (SDR)  was  conducted.  [Ref.  52]  The  System  Requirements 
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Review  (SRR)  was  held  in  December  1996,  and  was  used  to  establish  the  first 
System/Segment  Specification  (S/SS)  for  the  AAAV(P)  and  the  AAAV(C).  Changes  to 
requirements  in  the  specification  are  conducted  on  an  ongoing  basis. 

General  Dynamics  also  uses  data  base  matrices  to  track  both  the  requirements 
validation  process  and  the  test  planning  process.  The  Requirements  Validation  Matrix 
identifies  the  S/SS  by  page  and  number,  identifies  the  responsible  IPT,  lists  the  type  of 
validation  method  (Inspect,  Analyze,  Demonstrate,  Model/Simulate,  Test),  identifies  the 
level  of  validation  (Component,  Subsystem,  System),  and  identifies  vehicle  weight 
configuration  and  environmental  condition  for  testing.  [Ref.  52]  The  Test  and  Evaluation 
Validation  Matrix  has  similar  content  to  the  Requirements  Validation  Matrix,  but 
contains  greater  detail  on  environmental  conditions  for  testing.  The  Test  and  Evaluation 
Integrated  Test  Plan  was  combined  by  General  Dynamics  into  an  Access  Data  Base  that 
provides  linkage  between  the  Work  Breakdown  Structure  (WBS),  the  Systems 
Engineering  Design  Schedule  (SEDS),  and  the  S/SS.  [Ref.  90]  During  the  System 
Design  Review,  the  contractor  briefed  that,  “This  data  base  is  considered  to  be  the  most 
thorough,  detailed,  cross-linked  and  user  friendly  resource  for  the  Integrated  Test  Plan 
CDRL  -T016.”  [Ref.  90] 

F.  CHAPTER  SUMMARY 

This  chapter  established  and  documented  the  eleven-year  evolution  of  the  AAAV 
PMO's  test  and  evaluation  strategy  from  Milestone  0  to  the  present,  and  the  anticipated 
test  and  evaluation  strategy  for  the  future  by  analyzing  three  versions  of  the  program’s 
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Test  and  Evaluation  Master  Plan.  This  chapter  provides  the  foundation  for  the 
development  and  analysis  of  test  and  evaluation  issues  faced  by  the  PMO. 
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VI.  TEST  AND  EVALUATION  RELATED  ISSUES  AND  LESSONS  LEARNED 
A.  INTRODUCTION 

This  chapter  discusses  issues  facing  the  AAAV  Program  Management  Office  that 
have  had,  or  will  have,  impact  on  the  development  and  conduct  of  the  program’s  test  and 
evaluation  strategy.  Where  available,  the  courses  of  action  (COAs)  available  to  the  PMO, 
and  the  associated  advantages  and  disadvantages  of  the  COAs,  are  discussed  in  order  to 
provide  potential  guidance  for  other  program’s  facing  similar  issues.  Applicable  lessons 
learned  are  then  developed  for  each  issue.  Table  7  outlines  the  specific  issues  discussed. 


Test  Management 

Issue:  What  is  the  potential  impact  of  down-selection  and  co-location  on  the  ability  of  the  contractor  to 
provide  adequate  test  and  evaluation  support  to  the  design  and  development  of  the  AAAV? 

Issue:  What  working  relationship  between  oversight  agencies,  external  agencies,  and  the  DRPM  best 
serves  the  PMO’s  test  program? 

Developmental  Test  and  Evaluation  ~”1 

Issue:  What  are  the  effects  of  schedule  compression  on  the  DRPM’s  ability  to  complete  all  scheduled  DT 
test  objectives  prior  to  the  AAAV(P)  EOA  and  Milestone  II? 

Issue:  Who  is  best  suited  to  develop  Detailed  Test  Plans? 

Issue:  What  is  appropriate  level  of  detail  necessary  for  Detailed  Test  Plans? 

Operational  Test  and  Evaluation 

Issue:  What  is  a  sufficient  level  of  operational  assessment  prior  to  the  conduct  of  IOT&E? 

Issue:  What  operational  issues  are  truly  critical? 

Issue:  What  is  the  impact  on  test  planning  for  IOT&E  if  LRIP  vehicles  are  no  longer  available? 

Issue:  How  can  the  DRPM  best  meet  FOF  RTCA  requirements?  1 

Issue:  What  if  OT  events  prove  system  fails  to  meet  operational  effectiveness  criteria? 

Issue:  Will  USMC  structure  be  in  place  to  properly  assess  supportability  issues  with  system  deployment 
during  IOT&E? 

Live  Fire  Test  and  Evaluation  < 

Issue:  How  can  an  adequate  Vulnerability  Assessment  be  conducted  for  the  AAAV(C)  variant  without  a 
Full  Up  System  Level  (FUSL)  Test  Event? 

Issue:  What  key  areas  of  concern  face  a  Program  Management  Office  when  developing  a  Live  Fire  Test 
and  Evaluation  Strategy? 

Requirements 

Issue:  How  does  the  DRPM’s  “stop  work”  decision  on  the  AAAV(C)  due  to  changes  in  the  system’s 
requirements  affect  the  test  strategy? 

Issue:  How  does  the  DRPM  best  address  the  interoperability  KPP  requirement  in  the  Test  and  Evaluation 
Master  Plan? 

Table  7.  Summary  of  AAAV  Test  and  Evaluation  Issues. 
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B.  TEST  MANAGEMENT 


Issues  facing  the  PMO  at  the  test  management  level  fall  into  two  general 
categories.  First,  the  working  relationship  between  the  contractor  and  the  government 
was  significantly  effected  by  the  decision  to  down-select  to  one  contractor  in  the  PDRR 
Phase,  and  then  to  co-locate  the  DRPM  and  the  contractor  in  the  same  facility.  Second, 
the  working  relationship  between  the  PMO,  oversight  agencies,  and  external  agencies 
was  effected  by  the  DRPM’s  decision  to  “activity  engage”  each  in  the  decision  making 
process. 

1.  Down  Selection  and  Co-location 

Issue:  What  is  the  potential  impact  of  down-selection  and  co-location  on  the 
ability  of  the  contractor  to  provide  adequate  test  and  evaluation  support  to  the  design 
and  development  of  the  AAA  V? 

Discussion:  The  ability  of  the  contractor  to  support  test  and  evaluation  activities 
is  based  on  the  number  and  caliber  of  test  personnel  assigned  to  the  program.  The 
contractor  has  many  options  in  a  co-location  environment  including:  determining  which 
specific  skill  levels  and  personnel  are  to  be  assigned  and  relocated  to  the  new 
facility/project;  determining  the  number  of  test  personnel  to  be  assigned  and  relocated; 
and  determining  the  level  of  matrix  support  to  be  provided  from  the  headquarters  location 
to  the  new  project’s  facility.  “The  co-location  decision  must  account  for  segmenting  the 
contractor’s  test  capabilities.”  [Ref.  91] 

Contractor  personnel  indicated  that  when  the  PDRR  contract  was  awarded  to  GD 
Land  Systems,  GD  was  forced  to  split  out  a  slice  of  their  test  personnel  from  the  Sterling 


152 


Heights,  Michigan  headquarters  to  send  to  the  newly  created  GDAMS  division  in 
Woodbridge,  Virginia.  Methods  available  to  the  contractor  to  make  personnel  relocation 
decisions  include  asking  for  volunteers,  directing  personnel  to  relocate,  or  a  combination 
of  volunteering  and  forced  relocation.  Concerns  with  these  methods  include  meeting  the 
required  number  of  test  personnel  willing  to  relocate,  meeting  the  required  skill  levels  of 
test  personnel  willing  to  relocate,  and  hiring  new  test  personnel  to  meet  possible 
manpower  deficiencies  resulting  from  those  not  willing  to  relocate.  Contractor  personnel 
also  indicated  that  the  potential  exists  for  Management  to  underestimate  the  level  of  test 
experience  and  diversity  of  test  personnel  backgrounds  necessary  for  a  system  of  this 
scale  and  complexity.  [Ref.  91] 

Another  concern  of  co-location  is  that  Contractor  Management  may  not  properly 
assess  the  manning  level  of  test  personnel  required  for  the  project.  The  Test  Manager  for 
GDAMS  stated  that  he  initially  estimated  needing  at  least  three  personnel  for  testing,  yet 
“management  estimated  only  half  a  person.”  [Ref.  92]  The  present  GDAMS  test  staffing 
of  four  engineers  is  now  seen  by  the  test  manager  as  too  few.  His  main  points  are  that  the 
original  estimates  of  contractor  test  personnel  did  not  account  for  the  IPT  process,  and  the 
concurrent  testing  of  three  prototypes.  “Because  test  representation  is  required  in  all  the 
subsystem  IPTs,  test  section  manning  levels  must  account  for  the  IPT  process.  With  a 
small  staff,  test  personnel  can  often  find  themselves  attending  so  many  IPT  meetings  that 
the  time  needed  for  test  planning  and  execution  becomes  greatly  reduced.”  [Ref.  92] 
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The  number  of  contractor  test  personnel  also  influences  the  amount  of  testing  that 
can  occur  concurrently.  With  the  addition  of  P2  and  P3,  the  contractor’s  test  manager 
believes  he  is  reaching  the  limits  of  his  capabilities  to  setup,  execute,  and  collect  data  at 
multiple  locations  with  the  different  vehicles  conducting  different  tests.  A  solution  or  aid 
to  this  potential  problem  is  that  the  Government  has  established  a  small  staff  of 
Development  Test  (DT)  Marines  to  conduct  testing  with  contractor  personnel.  The 
present  concept  is  for  the  contractor  to  provide  two  crewmen  and  the  Government  to 
provide  two  crewmen  per  prototype  during  each  testing  event.  The  contractor’s  senior 
crewmember  would  serve  as  crewchief  of  the  test  vehicle.  This  provides  the  contractor 
the  ability  to  staff  a  single  test  shift  on  a  six-day-a-week  basis.  However,  this  team  effort 
would  only  allow  the  contractor  to  staff  a  double  shift  for  up  to  two  weeks.  At  that  point, 
fatigue  becomes  a  safety  factor  and  testing  would  have  to  return  to  a  single  shift.  [Ref. 
92] 

A  specific  issue  for  amphibious  vehicle  testing  is  that  both  contractor  and 
Government  crewmembers  must  be  scuba  dive  qualified  for  water  testing  safety  purposes. 
This  requirement  impacts  manpower  decisions  with  respect  to  the  number  of  qualified 
personnel  available  to  conduct  concurrent  testing.  [Ref.  92] 

Lesson:  A  co-location  decision  must  account  for  the  segmenting  of  the 
contractor’s  test  capabilities.  Both  program  management  and  contractor  management 
must  properly  assess  the  number,  experience,  and  skill  level  of  test  personnel  required  for 
projects  based  on  the  level  of  system  complexity.  Test  section  manning  levels  must  also 
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account  for  the  IPT  process,  the  number  of  test  articles,  and  the  level  of  concurrent  testing 
to  occur  at  different  test  sites.  Test  representation  will  be  required  in  all  subsystem  level 
IPTs.  [Ref.  92] 

Lesson:  The  IPT  process  and  co-location  have  been  beneficial  in  the  conduct  of 
daily  communications  with  Government  counterparts.  Neither  the  implementation  of 
ongoing  schedule  changes,  nor  the  updating  of  the  test  plans  in  a  rapid  manner  would 
have  occurred  without  co-location.  [Ref.  92] 

2.  Working  Relationships 

Issue:  What  working  relationship  between  oversight  agencies,  external  agencies, 
and  the  DRPM  best  serves  the  PMO’s  test  program? 

Discussion:  The  DRPM  faced  two  general  options  with  respect  to  the  PMO’s 
overall  relationship  with  oversight  agencies  and  other  external  agencies.  From  a  test  and 
evaluation  perspective  oversight  agencies  include  DOT&E,  DOT&E  (LFT),  and  DS&TS 
(DT).  External  agencies  include  MCCDC,  MCOTEA,  and  Test  Sites  such  as  ATC  and 
AVTB.  Option  One  would  be  to  maintain  a  “report  as  required”  relationship  in  which  the 
dialog  between  the  PMO,  the  oversight  agencies,  and  the  external  agencies  would  be  kept 
to  an  essential  minimum.  Option  Two  would  be  to  “fully  engage”  the  agencies  while 
maintaining  an  up-front,  forthright,  and  proactive  relationship.  [Ref.  93] 

The  DRPM  has  chosen  to  actively  engage  both  the  oversight  agencies  and  the 
external  agencies  as  early  in  the  test  planning  process  as  possible.  He  believes  that  this 
inclusiveness  better  serves  to  accomplish  the  program  office’s  mission  while  still  meeting 
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the  other  agency’s  requirements.  Specifically,  both  sides  can  reach  a  balanced  and 
acceptable  level  of  testing  with  meaningful  test  objectives  within  the  realities  of  the 
program’s  fiscal  constraints.  [Ref.  94] 

Lesson:  A  working  relationship  should  be  developed  and  maintained  that 
“actively  engages”  both  the  oversight  agencies  and  the  external  agencies  as  early  in  the 
test  planning  process  as  possible.  By  gaining  a  mutual  understanding  and  using  the 
agencies’  knowledge  and  expertise,  the  program  can  achieve  a  greater  efficiency  in  its  use 
of  resources.  To  achieve  this  efficiency,  every  effort  must  be  made  by  both  Government 
and  Contractor  personnel  to  optimize  their  time  and  resources  to  ensure  an  effective  test 
program.  This  optimization  is  a  mindset,  and  requires  continually  assessing  the  system’s 
requirements  and  the  test  objectives  to  ensure  that  redundancies  are  justified  or 
eliminated.  True  optimization  can  only  occur  if  oversight  and  external  agencies  are 
“actively  engaged”  early  in  the  process.  [Ref.  94] 

3.  Other  Test  Management  Lessons  Learned 

Lesson:  The  acquisition  reform  efforts  of  down  selection  to  one  contractor  in  the 
PDRR  phase,  co-location  of  the  Government  and  the  Contractor  in  the  same  facility,  and 
implementation  of  the  IPPD/ITP  process  have  proven  invaluable  to  the  test  planning 
process.  The  first  two  reform  measures  link  directly  to  the  latter,  and  because  they  are 
linked  they  result  in  a  synergistic  effect  for  the  program.  Without  these  combined  reform 
measures  the  program  would  not  have  been  successful  in  planning,  coordinating,  and 
conducting  other  reform  measures  that  meet  the  DRPM’s  optimization  goals.  Integrated 
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Contractor  and  Government  developmental  testing  could  not  have  been  coordinated, 
Detailed  Test  Plans  written  by  the  Contractor  would  not  have  been  acceptable  to  the  test- 
site  agencies,  and  generation  of  the  Test  and  Evaluation  Management  Plan  and  the  Live 
Fire  Test  and  Evaluation  Management  Plan  would  have  been  unacceptably  delayed.  [Ref. 
93] 


Lesson:  The  Test  and  Evaluation  Working  Level  IPT  is  essential  for  the 
successful  implementation  of  the  program’s  Test  and  Evaluation  Strategy.  The  T&E 
WIPT  is  a  value  added  management  tool  that  enables  enhanced  communication,  ensures 
mutual  understanding,  and  prevents  confusion  among  the  critical  decision  makers  at  the 
appropriate  level.  The  development  of  the  subordinate  LFT&E  Working  Group  has 
proven  successful  in  meeting  OSD  oversight  requirements.  Consideration  was  made  to 
have  similar  subordinate  Working  Groups  for  Developmental  Testing  and  Operational 
Testing,  but  the  T&E  WIPT  determined  that  these  Working  Groups  might  actually 
decrease  effectiveness.  Regardless,  each  Program  Management  Office  must  determine 
the  most  appropriate  Test  and  Evaluation  management  plan  based  on  that  program’s 
unique  needs.  [Ref.  97] 

Lesson:  The  emphasis  on  “empowering”  IPT  members  and  making  decisions  at 
the  lowest  possible  level  makes  it  difficult  to  track  the  decision-making  process  regarding 
some  issues.  Unless  the  IPT  develops  and  maintains  an  adequate  record  of  the  IPT’s 
discourse  during  a  meeting,  the  multitude  of  courses  of  action  (COAs)  analyzed  and 
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decisions  made  can  not  be  traced,  reviewed,  or  analyzed  over  any  period  of  time. 
Therefore,  a  record  of  the  decisions  and  COA  analysis  should  be  maintained  for  future 
traceability  and  reference.  The  IPT  group  analysis  and  consensus  decision-making 
process  differs  significantly  from  the  more  formal  “staff  planning  process”  used  by  most 
military  organizations.  Unless  issues  are  raised  to  the  DRPM  level,  seldom  are  COAs  for 
those  issues  documented,  developed  in  their  entirety,  and  presented  as  such  for  an 
eventual  decision.  Military  members  assigned  to  a  program  office  need  to  understand  the 
difference  between  the  IPT  decision-making  process  and  the  more  traditional  “staff 
planning  process.” 

Lesson:  The  congressional  funding  plus-ups,  the  increased  procurement  of  PDRR 
prototypes,  and  the  schedule  acceleration  concessions  are  out  of  the  test  manager’s 
control,  yet  each  had  a  significant  impact  on  his  ability  to  plan,  coordinate,  and  prepare 
for  the  conduct  of  the  AAAV’s  test  program.  The  DRPM’s  test  manager  was  able  to 
overcome  these  issues  because  of  his  understanding  of  the  DRPM’s  test  strategy  and  his 
relationship  with  other  agencies.  Thoroughly  understanding  the  DRPM’s  envisioned  end- 
state  for  his  testing  program  has  allowed  the  test  manager  to  make  necessary  short-term 
changes  to  the  test  program  based  on  changing  environmental  conditions.  Additionally, 
the  test  manager  had  a  well  founded  working  relationship  with  the  contractor’s  test 
manager,  the  OSD  oversight  representatives,  the  requirements  officer,  the  test-site 
representatives,  and  the  independent  operational  test  director.  The  combination  of  a  clear 
test  strategy  and  good  working  relationships  placed  the  test  manager  in  a  better  position 
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to  maintain  an  awareness  of  the  program’s  overall  test  status,  make  necessary  changes  to 
the  test  plan,  manage  the  test  program,  and  achieve  the  system’s  test  objectives. 

Lesson:  The  updating  of  the  AAAV’s  1995  TEMP  to  meet  the  needs  of  the 
Milestone  II  DAB  review  was  a  long  and  complex  process.  As  the  program  moves  from 
the  PDRR  Phase  the  to  EMD  Phase,  the  level  of  detail  in  the  Draft  TEMP  increased 
significantly  based  on  the  system’s  increased  maturity  level  and  the  increased  proximity 
to  actual  test  events.  The  PMO  has  been  successful  in  generating  an  updated  TEMP 
because  of  four  key  decisions.  First,  the  program  convened  the  process  as  early  as 
possible.  Second,  the  program  actively  sought  input  from  all  OSD  oversight  agencies 
especially  with  respect  to  Live  Fire  Test  and  Evaluation.  Third,  the  program  worked 
closely  with  the  requirements  developer  to  ensure  that  the  ORD  update  process  paralleled 
the  TEMP  update  ensuring  continuity  between  the  two  documents.  Finally,  the  program 
maintained  close  and  constant  coordination  with  the  independent  operational  test  director 
from  MCOTEA.  This  coordination  allowed  MCOTEA  and  the  DRPM  to  develop  Part  IV 
of  the  TEMP  with  a  significant  level  of  continuity.  By  implementing  these  four 
decisions,  a  program  management  office  can  reduce  the  complexity  and  the  time 
necessary  to  make  TEMP  revisions. 

C.  DEVELOPMENTAL  TEST  AND  EVALUATION 

The  two  main  areas  of  issue  facing  the  DRPM  with  respect  to  Developmental  Test 
and  Evaluation  include  schedule  constraints  that  impact  the  amount  of  time  available  to 
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achieve  test  objectives,  and  determining  the  appropriate  level  of  responsibility  for 
generating  Detailed  Test  Plans. 

1.  Schedule 

Congressional  funding  plus-ups  have  generally  been  attached  to  concessions  by 
the  PMO  to  accelerate  the  program’s  schedule.  Specific  funding  enhancements  in  FY97 
and  FY98  were  tied  to  the  movement  forward  of  Milestone  II,  Milestone  III,  and  IOC. 
More  recently,  the  PMO  has  sought  approval  to  move  Milestone  II  forward  from  January 
2001  to  November  2000  in  order  to  reduce  the  impact  of  possible  delays  on  the  program’s 
progress  caused  by  the  Presidential  election  and  political  appointment  process. 
Additionally,  the  first  prototype’s  (PI)  readiness  to  test  has  been  delayed  by  system 
integration  problems. 

Issue:  What  are  the  effects  of  schedule  compression  on  the  DRPM’s  ability  to 
complete  all  scheduled  DT  test  objectives  prior  to  the  AAA  V(P)  EOA  and  Milestone  II? 

Discussion:  In  February  1997,  the  Operations  Officer  conducted  a  T&E  Schedule 
Evaluation  brief  for  the  DRPM  with  the  expressed  purpose  “to  determine  where  3-4 
months  can  be  saved  between  Prototype  #1  delivery  and  Milestone  II.”  [Ref.  101] 
Assumptions  made  at  this  brief  included  delivery  of  PI  on  time  with  minimal  contractor 
shakedown  issues,  saving  time  by  conducting  concurrent  contractor  and  government 
developmental  testing,  the  availability  of  OT  personnel  during  the  conduct  of  DT  events, 
and  early  DOT&E  input  allowing  reduction  of  the  reporting  time  to  under  180  days.  [Ref. 
101] 
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The  DRPM  approved  this  compression  of  the  test  schedule  despite  increasing  the 
anticipated  level  of  risk.  Risk  was  assessed  as  “high”  regarding  PI  delivery  and 
shakedown,  and  “medium”  concerning  the  reduced  amount  of  time  to  conduct  the  EOA 
even  with  concurrent  DT/OT  events  and  two  prototypes.  [Ref.  101] 

As  a  result,  the  Contractor’s  Test  Manager  first  expressed  concerns  about 
insufficient  time  on  the  schedule  for  completing  test  objectives  in  October  1997  when  test 
schedule  risk  was  again  assessed.  Because  Contractor  and  Government  developmental 
testing  would  be  integrated,  the  test  manager  indicated  that,  “This  will  allow  a  reduction 
in  the  impact  to  the  overall  schedule  that  problems  would  [create].”  [Ref.  98] 

In  January  1998,  the  Contractor’s  Test  Manager  indicated  that  the  addition  of  the 
second  and  third  prototypes  should  ease,  “Our  ability  to  complete  all  test  objectives  in  the 
scheduled  amount  of  time...”  [Ref.  98]  The  Contractor’s  Test  Manager  also  stated  that, 
“The  third  vehicle  gives  us  the  flexibility  to  increase  testing  in  areas  that  are  behind 
schedule,  or  where  we  have  increased  concern”  [Ref.  98]  By  February  1999,  the 
contractor  had  developed  single-  and  double-shift  test  plans  for  test  delay  contingencies. 
“This  escalation  of  resource  utilization  has  been  added  to  our  planning  schedule  and  has 
been  briefed  to  the  Product  Design  Team  and  the  [DRPM].”  [Ref.  98] 

The  9  April  1999  virtual  design  database  (VDD)  test-schedule  risk  entry  by  the 
contractor  states: 

The  ability  to  successfully  complete  the  required  testing  on  time  for  the 
DAB  review  remains  as  before.  P-1  is  still  scheduled  for  a  2  August 
delivery  and  the  current  and  proposed  updated  schedules  show  that  the 
assigned  tasks  are  do-able  within  the  time  frame  allocated.  There  is  a 
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problem  with  P-2  delivery  it  has  been  moved... to  mid  November  and  it 
forces  that  portion  of  the  testing  into  a  tighter  time  frame  than  we  would 
like.  The  new  schedule  does  show  that  it  is  possible  to  complete,  but  it 
allows  even  less  “breathing  room”  than  previously.  [Ref.  98] 

In  late  1998  and  again  in  early  1999,  the  Contractor’s  Test  Manager  briefed  his 

concerns  about  the  test  schedule  to  the  DRPM.  He  described  the  existing  DT  schedule  as 

“too  tight”  and  a  sure  “formula  for  failure.”  [Refs.  99,  100]  He  emphasized  that  the  six- 

days  per  week  and  ten-hours  per  day  schedule  was  necessary  even  before  adding  in  any 

proposed  DT/OT  integration,  and  that  there  was  a  need  to  prioritize  testing  requirements 

should  the  schedule  not  allow  completion  of  all  planned  test  objectives.  In  order  to 

resolve  this  concern  the  Contractor’s  Test  Manger  proposed  the  following  [Ref.  99]: 

•  Look  for  testing  that  can  otherwise  be  moved  beyond  the  EOA  to  increase 
schedule  flexibility. 

•  Identify  DT/OT  integration  possibilities,  especially  for  the  KPPs  and 
Milestone  II  exit  criteria. 

•  Ensure  enough  flex  time  remains  in  the  schedule  to  account  for  “Murphy’s 
Law.” 

During  an  interview  with  the  author,  the  Contractor’s  Test  Manager  stated  that  his 
two  primary  concerns  with  the  existing  test  schedule  are  a  lack  of  built-in  maintenance 
time,  and  an  underestimation  of  the  time  necessary  to  complete  full-up  system 
integration.  First,  the  Test  Manger  believes  the  present  schedule  lacks  a  buffer  at  the  end 
of  each  test  event  to  allow  for  the  accomplishment  of  necessary  corrective  maintenance  to 
the  system.  He  feels  the  schedule  is  based  on  the  assumption  that  the  system  will  perform 
without  developing  faults  or  problems.  He  indicated  that,  based  on  his  experience, 
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between  50%  to  60%  of  the  schedule  needs  to  be  allotted  to  maintenance  time  in  order  to 
make  system  and  subsystem  repairs  during  the  conduct  of  DT.  He  also  indicated  that, 
based  on  his  experience,  most  PM’s  are  not  willing  to  build  that  into  the  schedule,  or  if 
built  into  the  schedule,  the  maintenance  time  is  seen  as  expendable  when  the  program  is 
behind  schedule  and  trying  to  find  time  to  catch  up.  [Ref.  92] 

Second,  the  Contractor’s  Test  Manager  stated  that  full-up  system  integration 
“always  takes  longer”  than  planned  for  or  anticipated.  His  concern  is  that  despite  the 
program’s  experience  with  the  ATR  and  the  HTR,  the  schedule  still  does  not  account  for 
the  unknown  factors.  “The  AAAV  is  a  revolutionary  upgrade  in  system  performance. 
This  aspect  of  the  system  even  further  complicates  the  integration  process  and  increases 
possibility  for  problems  in  testing.”  [Ref.  92]  He  further  indicated  that  the  revolutionary 
aspect  of  the  system’s  development  should  be  accounted  for  in  the  schedule  by  adding 
more  maintenance  time.  [Ref.  92] 

An  additional  concern  expressed  by  the  Contractor’s  Test  Manager  is  that  the 
political  justification  for  moving  up  Milestone  II  from  January  2001  to  Nov  2000  only 
compounds  the  lack  of  system  readiness  for  testing  at  this  time.  As  a  point  of  summary 
the  Test  Manager  said,  “The  Milestone  keeps  moving  to  the  left  and  the  system’s 
readiness  for  testing  keeps  slipping  to  the  right.  Eventually  something  will  have  to  give.” 
[Ref.  92] 

The  DRPM’s  Test  Manger  stated  that  he  has  already  lost  six  man-months  of  testing 
due  to  the  system’s  lack  of  readiness  at  scheduled  test  periods.  He  also  indicated  that  the 
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DT  test  schedule  was  designed  to  accommodate  a  certain  amount  of  slippage.  As  an 
example,  the  test  events  for  Ft.  Story,  Virginia  focused  on  testing  the  system’s  ability  to 
achieve  high  water  speeds  in  Sea  State  2  or  above.  Knowing  that  ocean  conditions  on  the 
east  coast  and  west  coast  differ  significantly,  the  program’s  original  goal  was  to  achieve 
the  high  water  speed  in  “East  Coast”  Sea  State  2,  and  then  be  able  to  compare  that  to 
“West  Coast”  Sea  State  2  at  Camp  Pendleton,  California.  At  this  time,  the  Ft.  Story  high 
water  speed  testing  has  been  cancelled  to  accommodate  schedule  changes,  and  the  high 
water  speed  testing  will  only  occur  at  Camp  Pendleton.  [Ref.  93] 

Working  closely  with  the  Contractor  Test  Manager,  the  DRPM  Test  Manger 
established  test  objective  priorities  that  would  ensure  a  sufficient  level  of  testing  prior  to 
both  the  scheduled  EOA  and  Milestone  II.  The  test  objective  priorities  are  as  follows 
[Ref.  93]: 

•  Product  Definition  and  Risk  Reduction  Phase  (PDRR)  exit  criteria. 

•  Governmental  Acceptance  Test  Criteria. 

•  Essential  Critical  Technical  Parameters  (CTPs)  to  verify  system  “safe  and 
ready”  for  the  EOA. 

•  Remaining  Critical  Technical  Parameters  (CTPs)  as  listed  in  the  TEMP. 

•  System/Subsystem  Specifications. 

The  time  necessary  for  test  agencies  to  complete  their  reporting  process  is  an 
additional  concern  that  impacts  the  available  time  prior  to  the  Milestone  II  decision.  A 
movement  of  Milestone  II  to  November  2000  would  not  provide  MCOTEA  the  requisite 
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120  days  to  complete  their  assessment  report  before  the  convening  of  the  DAB. 
Concessions  by  DOT&E,  MCOTEA,  and  the  DRPM  will  enable  the  DRPM  to  enter  the 
DAB  process  relying  on  the  “Quick  Look”  report  generated  by  MCOTEA.  Traditionally, 
the  Quick  Look  Report  is  considered  a  “close  hold”  report  in  which  there  is  limited 
distribution  to  MCOTEA,  MCCDC,  and  the  Assistant  Commandant  of  the  Marine  Corps. 
However,  DOT&E  as  agreed  to  accept  the  Quick  Look  Report  as  sufficient  enough  for 
the  Milestone  II  review.  In  order  to  achieve  this  rapid  of  a  turn  around  from  the 
completion  of  testing  to  the  generation  of  a  Quick  Look  Report,  DOT&E  and  MCOTEA 
have  agreed  to  conduct  concurrent  data  analysis.  [Ref.  78] 

Lesson:  In  order  to  meet  schedule  compression  and  optimization  goals,  all  areas 
of  testing  should  be  reviewed  early  and  with  all  agencies  involved.  The  results  of  the 
DRPM-AAA  schedule  review  included  establishing  an  integrated  Contractor  and 
Government  Developmental  Testing  program,  increasing  the  number  of  combined  DT 
and  OT  events,  and  streamlining  the  Live  Fire  Test  and  Evaluation  schedule.  These 
efforts  allowed  the  Program  Management  Office  to  initiate  an  accelerated  integrated 
schedule  without  significantly  increasing  schedule  or  performance  risk.  [Ref.  97]  Failure 
to  begin  test  schedule  reviews  and  involve  all  agencies  as  early  as  possible  could  result  in 
cost  overruns,  unacceptable  increases  in  risk,  and  an  inability  to  meet  test  schedules. 
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2.  Detailed  Test  Plans 


The  two  issues  facing  the  DRPM  with  regard  to  Detailed  Test  plans  are 
identifying  who  is  in  the  best  position  to  develop  the  DTP,  and  what  level  of  detail  should 
the  DTP  contain  to  meet  the  requirements  of  Development  Testing  for  the  system. 

Issue:  Who  is  best  suited  to  develop  Detailed  Test  Plans? 

Discussion:  In  a  traditional  developmental  testing  program,  the  Contractor  would 
develop  DTPs  for  their  portion  of  Contractor  specific  DT.  The  PMO  would  then  be 
responsible  for  developing  DTPs  for  the  Government  specific  DT.  Generally,  the  PMO 
would  delineate  the  actual  writing  of  the  DTP,  or  portions  of  the  DTP,  to  the  test-site 
agency  as  part  of  the  contract  for  them  to  conduct  DT  test  events.  As  an  example,  ATC 
would  write  the  DTP  for  their  portion  of  Land  Mobility  testing,  and  AVTB  would  write 
the  DTP  for  their  portion  of  Water  Mobility  testing.  [Ref.  78] 

In  this  case,  because  the  DRPM  and  GDAMS  are  conducting  integrated 
Contractor  and  Government  Developmental  Testing,  the  decision  was  made  to  have  the 
Contractor  write  the  entire  DTP  for  all  phases  of  the  DT-I  process.  The  result  was  a 
Detailed  Test  Plan  containing  Five  Books  that  exceed  1000  total  pages. 

Options  available  to  the  DRPM  included  following  the  “traditional  approach,” 
writing  the  DTP  “in-house,”  or  having  the  Contractor  write  the  DTP  as  was  ultimately 
decided.  Considerations  to  account  for  when  making  this  decision  include  the  following: 

•  Number  of  Contractor  test  personnel  available  to  write  the  DTP. 

•  Experience  of  Contractor  test  personnel  with  developing  and  writing  DTPs. 
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•  Experience  of  PMO  test  personnel  with  developing  and  writing  DTPs 

•  Costs  associated  with  the  PMO  conducting  an  “in-house”  writing  of  the  DTP. 

•  Costs  associated  with  having  the  test-site  agency  write  the  DTP. 

•  Time  that  it  would  take  for  each  to  develop  a  DTP. 

•  Quality  of  the  DTP  as  developed  by  each. 

•  Ability  to  coordinate  the  development  of,  and  changes  to,  the  DTP  with 
regards  to  Contractor,  Government,  and  test-site  agency  locations  and  IPT 
processes. 

Lesson:  The  suitability  of  a  specific  organization  to  develop  a  program’s  Detailed 
Test  Plan  is  situationally  dependent,  and  requires  an  analysis  of  the  above  considerations. 
Regardless  of  the  final  decision  as  to  who  actually  writes  the  DTP,  the  Program  Manager 
remains  ultimately  responsible  for  the  success  of  the  system’s  developmental  testing 
program. 

Issue:  What  is  appropriate  level  of  detail  necessary  for  Detailed  Test  Plans? 

Discussion:  The  contractor’s  test  manager  stated  that  the  Government  required 
Detailed  Test  Plans  (DTPs)  for  DT-I  are  significantly  more  detailed  than  DTPs  he  has 
seen  used  for  other  programs.  The  AAAV’s  DTP  for  DT-I  currently  consists  of  five 
books  totaling  over  1000  pages.  [Ref.  86]  His  concern  is  that  the  DTPs  are  so  detailed 
that  changes  to  the  schedule  cause  the  contractor  to  make  complete  changes  to  the  DTP, 
where  as  a  less  detailed  plan  may  allow  for  greater  flexibility  when  schedule  slippage 
occurs.  The  contractor’s  test  manager  stated  that  the  Government  wanted  the  expected 
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level  of  detail  to  be  planned  to  the  day,  and  in  some  cases  to  the  hour,  even  though  the 
planned  testing  would  not  occur  for  almost  three  years.  “This  level  of  required  detail  so 
far  in  advance  leads  to  the  counter-productivity  of  test  personnel  and  their  time.”  [Ref. 
92] 

The  DRPM’s  Test  Manager  believes  that  the  DTPs  contain  a  “sufficient”  amount 
of  detail  for  the  conduct  of  integrated  Contractor  and  Government  development  testing. 
He  indicated,  however,  that  the  possibility  for  reducing  the  length  of  the  DTPs  existed  if 
the  commonality  between  the  five  books  had  been  separated  into  a  specific  executive 
summary  book.  The  level  of  redundancy  may  have  been  reduced  by  “30%”  had  a 
different  format  been  used.  The  amount  of  detail  was  considered  less  of  an  issue  by  the 
PMO  than  by  the  Contractor.  [Ref.  93] 

Lesson:  Efforts  to  identify  and  reduce  redundancies  in  the  AAAV’s  DT-I  DTP 
should  have  been  consistent  with  the  DRPM’s  efficiency  and  optimization  goals.  A 
Detailed  Test  Plan’s  quality  is  based  on  its  ability  to  efficiently  and  effectively  allow  test 
personnel  to  accomplish  test  objectives  and  not  on  its  length,  volume,  or  size.  Thus,  the 
appropriate  level  of  detail  necessary  for  a  DTP  is  dependent  upon  the  situation  of  each 
program. 

3.  Propulsion  System  Demonstrator  Lessons  Learned 

The  following  lessons  learned  were  identified  during  execution  and  review  of  the 
Propulsion  System  Demonstrator’s  (PSD’s)  developmental  testing  conducted  in  the 
Concept  Exploration  Phase.  The  lessons  are  drawn  directly  from  the  June  1994  PSD  test 
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report  titled,  Test  and  Evaluation  of  the  Propulsion  System  Demonstrator.  They 


represent,  “insights  into  what  could  be  improved  or  done  differently  in  future  amphibious 
vehicle  development  programs... [and]  to  assist  in  the  design,  fabrication,  and  testing  of 
future  advanced  amphibious  vehicles,  Marine  Corps  systems,  and  Naval  marine  craft.” 
[Ref.  65 :p.  99]  The  lessons  learned  are  also  applicable  to  other  major  defense  acquisition 
programs. 

Lesson:  “Predicting,  full  scale  amphibious  vehicle  performance,  especially  high 
water  speed  versions,  can  be  accomplished  two  ways  -  computer  modeling  and 
conducting  scaled  hydrodynamic  model  tests.  Computer  modeling  is  the  least  costly 
alternative,  but  does  not  yield  a  very  accurate  full  scale  performance  envelope.  Scaled 
model  tests  on  the  other  hand  are  slightly  more  expensive  and  can  accurately  predict 
certain  aspects  of  the  full  scale  vehicle’s  performance.  Some  areas  of  a  vehicle’s 
hydrodynamic  performance  can  not  be  accurately  predicted  using  either  computer  models 
or  model  testing.  The  general  lesson  here  is  that  hydrodynamic  experiments  using  scale 
models  are  recommended  for  accurately  predicting  certain  aspects  of  vehicle 
performance.  Some  aspects  such  as  spray  and  drag  which  are  harder  to  predict,  can  at 
least  be  identified  and  minimized  from  the  model  tests.  Some  effects  though  can  only  be 
repeated  during  full  scale  trials.”  [Ref.  65:pp.  1 12-113] 

Lesson:  “Developing  future  combat  vehicles  requires  a  very  comprehensive  and 
detailed  Management  Plan.  This  plan  should  contain  how  costs,  schedule,  manpower 
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allocations,  subcontracts,  contract  documentation,  and  other  aspects  of  the  development 
effort  are  to  be  managed,  tracked,  and  controlled.  The  PSD  program  was  a  follow-on 
program  to  both  the  ATR  and  the  HWSTD  development  efforts.  The  scope  of  PSD  was 
thought  to  be  similar  to  these  previous  projects  so  many  of  the  same  management  tools 
were  applied.  Unfortunately,  PSD  was  a  much  more  complex  program  because  of  the 
major  subcontractor  involvement,  the  more  critical  systems  integration  requirements,  and 
the  added  manpower  used  to  perform  design,  fabrication,  and  testing  tasks.  The  older 
management  tools  were  inadequate  for  handling  the  larger  and  more  visible  PSD 
program.  Planning  is  the  important  part  of  the  development  process.”  [Ref.  65  :p.  1 14] 

Lesson:  “Creating  a  team  environment  is  essential  to  getting  all  personnel  who 
are  involved  in  the  effort  working  effectively  and  smoothly.  The  same  people  who 
design  the  systems  should  follow  the  work  through  the  fabrication  phase  and  into  testing. 
Contractor  design  personnel  assigned  to  PSD  were  the  same  HWSTD  designers.  These 
people  were  involved  with  the  fabrication  and  assembly  of  the  hardware,  the  initial 
checkout,  and  were  even  available  for  counsel  during  testing.”  [Ref.  65  :p.  115] 

Lesson:  There  was  no  written  standard  operating  procedure  for  safe  conduct  of 
full  scale  high  speed  hydrodynamic  testing.  In  order  for  military  and  Government 
personnel  to  operate  the  PSD,  safety  procedures  and  certain  physical  requirements  must 
be  met.  Safety  Standard  Operating  Procedures  (SOPs)  should  be  prepared  that  parallel 
the  test  plan  and  sets  guidelines  and  requirements  for  each  hydrodynamic  test  being 
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performed  such  as:  safety  certification,  crew  and  test  personnel  training  level,  support  and 
safety  equipment,  and  safe  operating  criteria.  [Ref.  65  :p.  115] 

Lesson:  “No  matter  how  much  effort  is  put  into  the  planning  of  tests,  they  always 
take  longer  than  originally  thought.  There  is  nothing  that  can  be  done  to  [completely] 
eliminate  failures  and  ‘Murphy’s  Law.’  However,  flexible  test  plans,  contingency 
procedures,  and  ample  spare  parts,  resources,  and  personnel  can  minimize  their  impact  on 
the  test  schedule.”  [Ref.  65 :p.  115] 

Lesson:  “Future  vehicle  systems  must  be  designed  for  easy  operation, 
maintainability,  accessibility,  and  inspection  by  crew,  maintenance,  and  troop 
personnel... [Many]  areas  of  the  PSD  design  lacked  adequate  human  factors 
considerations.  The  lessons  learned  regarding  human  factors  were  to  always  design 
components  with  their  eventual  user  in  mind.”  Components  should  be  designed  for  easy 
operation  in  a  variety  of  environments,  under  different  conditions  and  under  various 
operational  scenarios.  [Ref.  65  :p.  116] 

4.  Other  Developmental  Test  and  Evaluation  Lessons  Learned 

Lesson:  Early  prioritization  of  Developmental  Test  events  is  critical  for  the 
conduct  of  the  program’s  Test  and  Evaluation  strategy.  Prioritization  of  Developmental 
Testing  events  allows  the  program,  the  oversight  agencies,  and  the  test-site  agencies  to 
remain  focused  on  the  purpose  and  objective  of  each  test  event,  and  allows  for  greater 
flexibility  in  the  conduct  of  testing  should  schedule  slippage  become  a  problem 
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D.  OPERATIONAL  TEST  AND  EVALUATION 

Issues  facing  the  DRPM  with  respect  to  Operational  Test  and  Evaluation  include  the 
appropriate  level  of  system  assessment  prior  to  IOT&E,  the  development  of  Critical  Operational 
Issues,  the  impact  of  LRIP  vehicles  on  IOT&E,  the  appropriate  methods  to  conduct  FOF  RTCA, 
and  present  concerns  about  future  operational  testing. 

1.  Early  Operational  Assessments  /  Operational  Assessments 

The  1995  TEMP  indicated  that  once  EOA  4  was  completed  in  the  beginning  of 
FY02,  no  other  OT  event  would  occur  until  IOT&E  in  FY07. 

Issue:  What  is  a  sufficient  level  of  operational  assessment  prior  to  the  conduct  of 
IOT&E? 

Discussion:  While  reviewing  the  DRPM’s  1999  Draft  TEMP  in  preparation  for 
the  Milestone  II  DAB,  DOT&E  noted  that  the  five-year  lag  between  the  last  EOA  and 
IOT&E  was  unacceptable.  The  concern  was  that  the  program  would  be  unable  to  enter 
IOT&E  with  a  known  and  acceptable  level  of  risk  based  on  the  program’s  original 
schedule  calling  for  three  software  updates  and  one  hardware  update  following  EOA  4. 
Without  an  OT  event  to  track  progress  of  the  system’s  design  prior  to  IOT&E,  DOT&E 
deemed  the  risk  as,  “too  high.”  [Ref.  93] 

As  a  means  to  resolve  the  issue  DOT&E,  MCOTEA,  and  the  DRPM  developed  an 
Operational  Assessment  plan  that  included  adding  three  OA’s  to  the  EMD  Phase.  The 
OAs  would  serve  to  answer  operational  effectiveness  and  suitability  issues  for  extreme 
weather  and  lower  level  unit  command  and  control.  The  OAs  would  also  serve  as 
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preparation  and  rehearsal  for  MCOTEA  prior  to  IOT&E.  The  OAs  also  allows 
MCOTEA  and  the  DRPM  to  reduce  the  scope  of  IOT&E.  The  DRPM’s  Test  Manager 
believes  that  by  adding  the  OAs,  the  PMO  will  save  approximately  S36-85M  in  EMD 
funding.  [Ref.  93] 

The  second,  and  possibly  most  significant,  means  of  reducing  the  five-year  lag 
between  the  last  EOA  and  IOT&E  resulted  from  funding  and  program  schedule  issues 
more  so  than  test  issues.  The  trade-off  decision  by  the  PM  to  move  Milestone  III  and 
IOC  from  FY08  to  FY05  to  gain  additional  funding  inherently  compressed  the  time  span 
between  the  last  EOA  and  IOT&E  by  two  years. 

With  respect  to  test  specific  issues,  the  options  available  to  the  DRPM  included 
relying  on  the  EOA  reports  and  the  DT-II  results  to  determine  system  readiness  to  enter 
IOT&E,  conducting  a  limited  number  of  OAs,  or  conducting  the  three  OAs  as  finally 
determined.  The  disadvantage  of  not  conducting  OAs,  or  having  a  reduced  number  of 
OAs,  is  that  the  period  between  the  last  Operational  Assessment  and  IOT&E  was  still 
deemed  too  long,  and  the  associated  level  of  risk  remained  too  high.  An  advantage  to 
conducting  the  OAs  is  that  MCOTEA  will  be  better  prepared  to  support  and  conduct 
IOT&E.  Another  advantage  to  having  the  OAs  is  that  MCOTEA  will  gain  the 
opportunity  to  assess  the  Assault  Amphibian  School’s  ability  to  sufficiently  train  Marine 
crews  for  IOT&E.  MCOTEA  may  also  be  able  to  assess  the  school’s  ability  to  sustain 
sufficient  training  into  the  future  for  IOC  and  FOC. 
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Lesson:  The  use  of  multiple  EOAs  in  the  Concept  Exploration  Phase  and  the 
Program  Definition  and  Risk  Reduction  Phase,  and  the  use  of  OAs  in  the  Engineering  and 
Manufacturing  Phase  results  in  early  and  constant  user  input  to  system  design  and 
development.  Thus,  the  associated  performance  risk  as  the  system  enters  IOT&E  is 
significantly  reduced.  Five  years  between  a  final  EOA  and  IOT&E  without  an 
operational  assessment  may  raise  performance  risk  to  unacceptable  levels. 

2.  Critical  Operational  Issues  (COIs) 

The  1995  TEMP  lists  three  Operational  Effectiveness  COIs  and  five  Operational 
Suitability  COIs  for  the  generic  AAAV  system.  It  further  delineates  three  unique  COIs 
for  the  AAAV(P)  variant  and  two  COIs  unique  to  the  AAAV(C)  variant. 

Issue:  What  operational  issues  are  truly  critical? 

Discussion:  Though  sufficient  for  the  1995  TEMP,  OSD  directed  that  MCOTEA 
and  the  DRPM  review  and  change  those  COIs  for  the  1999  Draft  TEMP.  The  original 
COIs,  as  listed  in  the  1995  TEMP,  were  seen  by  DOT&E  as  not  distinguishing  between 
operational  issues  and  critical  operational  issues  (See  Chapter  V,  pp.  58-59).  As  the 
DRPM’s  Test  Manager  stated,  “Not  all  operational  issues  are  critical.”  [Ref.  93] 

In  order  to  review  the  intent  of  the  original  COIs  and  develop  more  meaningful 
COIs,  MCOTEA  established  a  temporary  IPT  in  May  and  June  of  1999  with  members 
from  the  DRPM,  MCCDC,  and  MCOTEA.  The  IPT  was  able  to  reduce  the  number  of 
AAAV  operational  issues  to  the  two  effectiveness  COIs  and  one  suitability  COI  listed  in 
Chapter  V,  page  60.  These  COIs  were  determined  to  be  truly  critical.  [Ref.  93] 
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Lesson:  The  use  of  an  IPT  to  determine  critical  operational  issues  is  successful 
when  representatives  from  requirements  development,  the  independent  operational  test 
activity,  the  PMO,  and  DOT&E  are  participating  members.  The  number  of  COIs  is  less 
important  than  the  relevance  of  the  COIs,  and  the  ability  to  generate  associated  Measures 
of  Effectiveness  and  Measures  of  Performance  needed  to  resolve  the  operational 
effectiveness  and  suitability  issues. 

3.  Test  Article  Availability 

Higher  level  funding  issues  dictated  that  the  LRIP  production  schedule  be 
adjusted.  First,  the  start  date  for  LRIP  production  was  delayed  as  a  concession  to  other 
Marine  Corps  programs.  Specifically,  the  LRIP  delay  allowed  the  Marine  Corps  to  take 
approximately  one  year’s  worth  of  AAAV  early  production  money  and  redistribute  it  to 
other  programs  allowing  those  programs  to  finish  their  annual  buy.  This  shift  of  funds 
reduced  the  overlap  of  multiple  programs’  funding  and  was  the  easiest  for  the  Marine 
Corps  to  accomplish.  Second,  the  DRPM  shifted  the  LRIP  start  date  to  a  later  date  to 
better  accommodate  a  more  stable  manpower  need  by  the  contractor.  Finally,  the  LRIP 
finish  date  was  moved  up  in  order  to  meet  the  newly  accelerated  program  schedule. 

Issue:  What  is  the  impact  on  test  planning  for  IOT&E  if  LRIP  vehicles  are  no 
longer  available? 

Discussion:  The  compressed  LRIP  schedule  resulted  in  a  dilemma  for  the 
DRPM.  The  four  LRIP  vehicles  previously  planned  for  use  in  the  IOT&E  FOF  RTCA 
phase  would  no  longer  be  available.  As  a  result,  the  DRPM  was  limited  to  two  basic 
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options.  First,  move  the  Milestone  III  DAB  review  to  a  later  period.  This  option  was 
seen  as  untenable  based  on  Congress’  desire  to  have  the  program  accelerate  its  integrated 
schedule.  The  second  option  was  a  combination  of  increasing  the  number  of  operational 
assessments  to  ensure  that  a  high  quality  vehicle  would  enter  IOT&E,  and  adding  an 
FOT&E  event  to  resolve  potential  operational  suitability  questions.  [Ref.  93] 

The  PMO  determined  that  having  eight  AAAV(P)  production  EMD  prototypes 
and  one  AAAV(C)  production  EMD  prototype  was  an  acceptable  combination  of 
vehicles  for  IOT&E  if  employed  as  a  mechanized  team  with  a  tank  platoon.  DOT&E 
concurred  that  the  mechanized  team  concept  would  be  sufficient  for  answering 
operational  effectiveness  questions,  but  was  not  convinced  that  operational  suitability 
questions  could  be  answered.  The  decision  to  add  an  FOT&E  event  for  LRIP  vehicles 
was  based  on  this  concern,  and  would  include  increasing  the  confidence  of  the  system’s 
reliability  and  availability.  [Ref.  93] 

Lesson:  Maturity  of  the  system’s  design  throughout  the  CE  Phase,  the  PDRR 
Phase,  and  into  the  EMD  Phase  effects  a  PM’s  ability  to  have  production  representative 
EMD  prototypes  available  for  IOT&E.  Programs  not  able  to  meet  the  same  level  of 
system  design  maturity  as  the  AAAV  early  in  the  system’s  lifecycle  may  not  be  able  to 
successfully  plan  for  or  execute  IOT&E  without  the  use  of  LRIP  vehicles.  A  lack  of 
LRIP  vehicles  could  impact  the  total  number  of  test  articles  available  for  use  in  IOT&E. 
A  reduced  number  of  test  articles  will  impact  IOT&E  planning  especially  if  the  number 
falls  below  a  doctrinal  unit  set. 
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4.  Force-On-Force  Real  Time  Casualty  Assessment 

Two  conditions  faced  the  DRPM  with  respect  to  Force-on-Force  (FOF)  Real  Time 
Casualty  Assessment  (RTCA).  First,  the  U.  S.  Army  lost  its  primary  FOF  RTCA  facility 
at  Ft.  Hunter-Liggett,  California  to  Base  Realignment  and  Closure  (BRAC)  decisions. 
Second,  DOT&E  had  stated  concerns  about  insufficient  FOF  RTCA  detail  in  their 
comments  about  the  DRPM’s  1995  TEMP.  “Prior  to  Milestone  II,  the  Marine  Corps 
must  develop  a  scoring  system  that  provides  an  effective  method  of  measuring  force  on 
force  evaluations  with  real  time  casualty  assessment  and  vehicle  removable  from  the 
battlefield.”  [Ref.  82] 

Issue:  How  can  the  DRPM  best  meet  FOF  RTCA  requirements? 

Discussion:  The  DRPM’s  Test  Manager  expressed  concerns  about  not  only 
meeting  the  FOF  RCTA  requirement,  but  also  how  to  meet  the  requirements  without 
incurring  significant  costs.  The  two  primary  alternatives  available  to  the  DRPM  included 
use  of  the  Army’s  National  Training  Center  (NTC)  facility  at  Ft  Irwin,  California,  or  use 
of  existing  Marine  Corps  Combined  Arms  Exercise  (CAX)  Program  controller  personnel. 
Aiding  in  the  DRPM’s  decision  was  DOT&E’s  view  that  the  Marine  Corps  should 
conduct  FOF  RTCA  in  a  way  that  would  gain  the  necessary  information  in  an 
environment  conducive  to  the  Marine  Corps’  capabilities.  As  a  result,  the  FOF  RTCA 
phase  of  IOT&E  will  be  conducted  at  29  Palms,  California  with  CAX  personnel 
providing  controller  and  evaluator  support  while  U.  S.  Army  Test  and  Experimentation 
Command  personnel  provide  planning,  executing,  scoring,  and  reporting  assistance.  The 
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Test  Manager’s  belief  is  that  a  Marine  Corps  run  FOF  RTCA  evolution,  though  more 

manpower  intensive  and  less  technologically  sophisticated  than  a  similar  Army  evolution, 

will  force  the  Marine  Corps  to  concentrate  on  critical  data  collection  requirements.  This 

concentration  of  effort  would  prevent  over-collection  of  unnecessary  data,  reduce 

associated  costs  of  conducting  an  FOF  RTCA,  and  provide  more  realistic  and  meaningful 

conclusions  about  the  AAAV’s  performance  in  a  force-on-force  environment.  [Ref.  93] 

Despite  the  compromise  between  sophistication  and  cost,  the  1999  Draft  TEMP 

still  states  the  following  concerns  with  respect  to  expense: 

The  lack  of  adequate  in-service  testing  capability  will  require  the  Marine 
Corps  to  rely  on  Army  testing  assets,  and  drive  up  testing  costs.  The 
prohibitive  cost  of  each  FOF  RTCA  evolution  will  also  limit  the  total 
number  of  OMPs  that  are  scored.  However,  the  number  of  AAAV  FOF 
RTCA  profiles  and  engagements  is  consistent  with  testing  for  the  Bradley 
Fighting  Vehicle,  and  should  therefore  provide  an  adequate  test.  [Ref. 

57:p.  IV-26] 

Lesson:  To  best  meet  FOF  RTCA  requirements,  a  close  working  relationship 
between  the  PM,  DOT&E  and  the  independent  operational  test  agency  should  be 
established  in  order  to  develop  the  scope  of  the  test,  to  determine  the  data  necessary  to 
conduct  FOF  RTCA  analysis,  and  to  optimize  the  data  collection  efforts.  Open  and 
continuous  communication  with  DOT&E  and  MCOTEA  will  allow  a  PMO  to  meet 
DOT&E  FOF  RTCA  requirements  while  minimizing  associated  costs. 
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5.  Potential  Operational  Issues 

The  following  two  items  were  expressed  as  concerns  by  the  Assistant  Program 
Manager  for  the  AAAV(P)  as  potential  issues  that  could  impact  the  PMO  based  on  the 
results  of  OT  events,  or  based  on  a  lack  of  preparedness  for  OT  events. 

Issue:  What  if  OT  events  prove  the  system  fails  to  meet  operational  effectiveness 
criteria? 

Discussion:  The  Assistant  Program  Manager  for  the  AAAV(P)  expressed  a 
concern  that  should  the  AAAV(P)  fail  to  meet  operational  effectiveness  issues,  then  the 
PMO  will  be  put  in  a  position  to  use  the  OT  event  results  as  a  basis  for  future  trade-off 
studies.  The  primary  area  of  performance  risk  that  the  program  faces  is  maintaining 
weight  at  an  acceptable  level  to  allow  the  vehicle  to  achieve  high  water  speed.  Though 
finally  ranked  as  "medium"  by  the  ONR,  exceeding  weight  predictions  was  a  consistent 
theme  throughout  each  of  their  three  technical  risk  assessments.  The  basis  for  the  trade¬ 
off  studies  would  be  which  subsystems  are  essential  and  which  subsystems  are  “nice  to 
have.”  [Ref.  97] 

Lesson:  Failure  to  meet  operational  effectiveness  during  OT  events  can  force  a 
program  into  system  requirement  trade-off  studies.  Anticipating  this  problem  and 
developing  pre-planned  courses  of  action  for  trade-off  analysis  can  minimize  the  overall 
impact  on  the  system’s  development.  Pre-planning  of  trade-off  analysis  should  include 
representatives  from  requirements  development  and  the  Fleet  to  ensure  maximum  user 
input  to  the  decision  making  process. 
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Issue:  Will  USMC  structure  be  in  place  to  properly  assess  supportability  issues 
with  system  deployment  during  IOT&E? 

Discussion:  The  second  concern  or  future  potential  issue  involves  system  support 
structure  readiness  for  IOT&E.  Specifically,  the  concern  is  whether  the  Marine  Corps 
will  have  sufficiently  evolved  its  concepts  of  Sea  Based  Logistics,  Over  the  Horizon 
doctrine  and  tactics,  and  two-echelon  maintenance  support  for  application  during  the 
IOT&E  of  the  AAAV.  [Ref.  97] 

Lesson:  Failure  to  have  these  concepts  fully  developed  and  ready  to  support 
IOT&E  could  lead  to  inaccurate  or  inconclusive  evaluation  of  the  system’s  operational 
effectiveness  and  suitability. 

6.  Other  Operational  Test  and  Evaluation  Lessons  Learned 

Lesson:  Early  prioritization  of  Operational  Test  events  is  critical  for  the  conduct 
of  the  program’s  Test  and  Evaluation  strategy.  Prioritization  of  Operational  Testing 
events  ensures  the  program,  the  independent  test  agency,  and  the  oversight  agencies  have 
common  expectations  of  the  OT  event.  This  is  especially  true  with  respect  to  Early 
Operational  Assessments.  The  program  is  responsible  for  ensuring  that  everyone  has 
common  expectations  about  the  assessments  from  the  EOA  event.  Common  expectations 
also  prevent  over  optimism  about  a  system’s  capabilities  during  the  conduct  of  the  EOA. 
Over  optimism  before  the  EOA  could  result  in  the  perception  by  oversight  agencies  that 
“poor  performance  in  an  EOA  equals  a  bad  program.”  [Ref.  97] 
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E.  LIVE  FIRE  TEST  AND  EVALUATION 

The  two  primary  issues  facing  the  DRPM  with  respect  to  Life  Fire  Test  and 
Evaluation  are  the  DOT&E  concerns  over  Live  Fire  Testing  of  the  AAAV(C)  variant  to 
support  a  comprehensive  Vulnerability  Assessment,  and  the  development  of  a  thorough 
and  sufficient  LFT&E  management  plan. 

1.  AAAV(C)  Full  Up  System  Level  (FUSL)  Testing 

Current  plans  for  FUSL  testing  of  the  AAAV  only  include  using  two  AAAV(P) 
production  representative  EMD  prototypes.  [Ref.  57:p.  IV-28]  The  1999  Draft  TEMP 
specifically  states,  “In  the  case  of  the  AAAV(C),  system  ballistic  performance  will  be 
leveraged  from  the  AAAV(P)  Full  Up  System  Level  (FUSL)  testing.”  [Ref.  57:p.  IV-17] 
The  Draft  TEMP  further  states,  “Assessment  of  the  AAAV(C)  crew  and  system 
survivability  will  be  made  through  applying  results  of  the  AAAV(P)  testing  and  results  of 
AAAV(C)  specific  component,  subsystem  and  controlled  damage  testing.”  [Ref.  57:p. 
IV-31] 

Issue:  How  can  an  adequate  Vulnerability  Assessment  be  conducted  for  the 
AAA  V(C)  variant  without  a  Full  Up  System  Level  (FUSL)  Test  Event? 

Discussion:  The  DRPM’s  Test  Manager  indicates  that  DOT&E  (LFT)  does  not 
believe  FUSL  testing  of  the  AAAV  (P)  satisfies  the  requirements  for  the  AAAV  (C)  Live 
Fire  testing.  DOT&E  (LFT)’s  view  is  that  the  Live  Fire  model  used  for  the  AAAV(P), 
and  thus  the  empirical  data  gained  from  that  model,  does  not  provide  a  thorough  enough 
vulnerability  assessment  for  the  AAAV(C).  DOT&E  (LFT)’s  primary  concern  is  not 


181 


with  penetration  effects  of  threat  weapon  systems  on  the  hull  of  the  AAAV(C),  but  with 
the  shock  attenuation  effects  through  the  vehicle’s  hull  and  the  subsequent  impact  on 
associated  communication  equipment.  As  a  result  of  these  concerns,  DOT&E  (LFT)  has 
expressed  that  they  would  not  necessarily  concur  with  the  PM’s  Vulnerability 
Assessment  Report  (VAR).  [Ref.  93] 

The  DRPM  is  presently  assessing  three  options  that  will  satisfy  DOT&E  (LFT) 
concerns.  Option  One  would  be  to  improve  the  AAAV(P)  variant’s  model  generated  data 
by  changing  or  altering  the  shots  and  shot  angles  against  the  AAAV(P).  These  new  shots 
and  shot  angles  would  be  determined  based  on  their  ability  to  generate  data  collection  for 
model  improvements.  Thus,  the  improved  model  would  allow  for  better  estimates  of  the 
AAAV(C)  variant’s  vulnerabilities.  The  advantage  to  this  option  is  that  even  if  model 
improvements  do  not  completely  assess  the  AAAV(C)’s  vulnerabilities,  the 
improvements  may  at  least  be  to  a  level  sufficient  to  satisfy  DOT&E  (LFT)  concerns.  It 
is  also  the  cheapest  of  the  three  options.  The  disadvantage  to  Option  One  is  that  the 
necessary  vulnerability  assessment  may  not  be  met  even  with  the  changes  to  shots  and 
shot  angles  on  the  AAAV(P).  [Ref.  93] 

Option  Two  would  be  to  conduct  non-penetrating  shock  testing  on  the  AAAV(C) 
pre-production  configuration  to  get  the  necessary  vulnerability  data.  The  advantage  to 
this  option  would  be  that  the  necessaiy  data  for  a  thorough  assessment  of  the  AAAV(C) 
variant’s  shock  vulnerabilities  could  be  collected.  The  primary  disadvantage  to  this 
option  is  that  this  Live  Fire  testing  would  occur  on  the  only  AAAV(C)  prior  to  IOT&E, 
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and  thus  if  the  AAAV(C)  sustained  damage  during  the  shock  testing,  it  would  be 
unavailable  for  IOT&E.  Another  disadvantage  is  the  increased  costs  associated  with 
conducting  non-penetrating  shock  testing  on  the  AAAV(C).  [Ref.  93] 

Option  Three  would  be  to  contract  and  build  another  AAAV(C)  production 
representative  prototype  that  would  then  undergo  complete  FUSL.  The  advantage  to  this 
option  is  that  it  provides  the  most  complete  vulnerability  assessment  of  the  AAAV(C). 
The  disadvantage  is  the  increased  cost  of  building  another  AAAV(C),  and  the  increased 
costs  associated  with  FUSL  testing.  It  remains  undetermined  at  this  time  if  the  AAAV(C) 
FUSL  would  be  in  addition  to  the  two  AAAV(P)’s  scheduled  for  FUSL,  or  in  place  of 
one  of  the  AAAV(P)s.  [Ref.  93] 

Lesson:  Vulnerability  Assessments  methods  for  Mission  Role  Variants  (MRVs) 
can  become  an  area  of  dispute  between  Live  Fire  Test  oversight  agencies  seeking  to  meet 
Congressional  directives  and  PM’s  seeking  to  reduce  program  cost  and  performance  risk. 
Determining  “adequate”  Vulnerability  Assessment  methods  for  a  specific  MRV  is 
dependent  upon  two  things.  First  is  the  degree  of  difference  between  the  primary  system 
undergoing  FUSL  and  the  designated  MRV.  Second,  adequacy  depends  on  the  ability  of 
the  PM  and  the  Live  Fire  Test  oversight  agency  to  reach  a  mutual  understanding  of 
system  vulnerabilities  in  a  realistic  operating  environment. 


183 


2.  Live  Fire  Test  and  Evaluation  (LFT&E)  Management  Planning 

The  DRPM  determined  that  the  best  means  to  document  and  manage  his  Live  Fire 
Test  and  Evaluation  (LFT&E)  program  was  to  outline  the  overall  plan  in  the  TEMP, 
develop  a  specific  LFT&E  Management  Plan  to  cover  LFT&E  issues,  and  have  a  separate 
and  subordinate  Survivability  Management  Plan  that  covered  specific  survivability 
issues. 


Issue:  What  key  areas  of  concern  face  a  Program  Management  Office  when 
developing  a  Live  Fire  Test  and  Evaluation  Strategy? 

Discussion:  Based  on  the  DRPM- AAA’s  experience  with  developing  a  separate 
LFT&E  Management  Plan,  six  areas  of  concern  face  the  PMO.  Specifically,  DOT&E 
and  DOT&E  (LFT)  identified  six  general  deficiencies  that  existed  in  the  second  draft  of 
the  DRPM’s  LFT&E  Management  Plan.  First,  the  LFT&E  Management  Plan  did  not 
describe  in  sufficient  enough  detail  the  test  schedule  of  major  test  events.  “The  current 
version  shows  only  bands  of  activity.  Without  such  a  schedule,  the  ability  to  execute  the 
plan  cannot  be  judged.”  [Ref.  102]  DOT&E’s  recommendation  was  to  add  detail  to  the 
schedule. 

Second,  “A  major  flaw  in  the  plan  is  that  it  treats  only  the  final  FUSL  phase  as  the 
LFT  required  by  Congress,  an  incorrect  inference  that  leads  to  some  strange  results.” 
[Ref.  102]  DOT&E’s  primary  concern  with  this  inference  is  the  delegation  of  duties 
throughout  the  Live  Fire  process.  Specifically,  the  DRPM  is  to  chair  the  LFT&E 
Working  Group,  a  subordinate  group  of  the  T&E  IPT,  for  all  LFT&E  testing  prior  to 
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FUSL.  Then,  MCOTEA  would  be  responsible  for  chairing  the  LFT&E  Working  Group 
during  FUSL.  DOT&E  states,  “As  independent  evaluator,  MCOTEA  should  chair  the 
working  group  during  the  development  and  approval  of  the  management  plan  and  during 
all  phases  of  test  execution  (as  is  the  case  for  Army  combat  vehicles).”  [Ref.  102] 

The  third  DOT&E  concern  is  that  the  LFT&E  Management  Plan  fails  to  address 
the  lethality  of  the  program’s  30mm  ammunition.  “The  quantities  of  ammunition  to  be 
procured  meet  the  threshold  for  LFT&E.”  [Ref.  102]  DOT&E’ s  recommendation  is  to 
add  appropriate  lethality  sections  to  both  the  LFT&E  Management  Plan  and  the  TEMP 
update. 

Fourth,  DOT&E  states  that  the  issues  of  susceptibility,  vulnerability,  and  Battle 
Damage  Assessment  and  Repair  (BDAR)  are  “too  focused  on  technical  requirements” 
and  not  on  operational  requirements.  [Ref.  102]  “As  written,  several  sub-issues  are  too 
requirement-specific,  and  they  mix  criteria  with  statements  of  issues.”  [Ref.  102] 
DOT&E  then  goes  on  to  give  a  recommended  set  of  vulnerability  and  BDAR  issues. 

The  fifth  area  of  concern  is  the  LFT&E  Management  Plan’s  lack  of  clarity  about 
roles  and  responsibilities  related  to  the  program’s  Modeling  and  Simulation  (M&S)  plan 
supporting  LFT&E.  The  draft  Charter  for  the  LFT&E  Working  Group  indicates  that 
GDAMS  will  have  responsibility  for  planning,  conducting,  and  reporting  the  results  of 
the  M&S  efforts  that  define  the  AAAV’s  vulnerability.  DOT&E  states  that,  “This  seems 
to  give  the  systems  contractor  a  major  role  in  assessing  the  vulnerability  of  his  own 
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product.”  [Ref.  102]  DOT&E  then  recommends  that  M&S  roles  and  responsibilities  be 

clearly  delineated  in  the  LFT&E  Management  Plan. 

The  final  concern  expressed  by  DOT&E  again  relates  to  GDAMS  role  in  LFT&E. 

The  LFT&E  Management  Plan  contains  a  draft  LFT&E  WG  charter  that 
gives  GDAMS  the  responsibility  to  plan,  conduct,  and  report  on  the 
validation  testing  of  various  AAAV  armors,  which  is  intended  to 
demonstrate  that  the  ballistic  requirements  of  the  armors  have  been  met. 

Unless  there  is  substantial  Government  oversight  of  this  test,  conflict  of 
interest  could  be  an  issue.  [Ref.  102] 

A  DOT&E  supporting  contractor  representative  from  the  Institute  for  Defense 
Analysis  (IDA)  provided  additional  comments  concerning  the  DRPM’s  Draft  LFT&E 
Management  Plan.  The  representative’s  primary  comments  focused  on  the  scope  and 
detail  of  the  plan,  and  on  LFT&E  Modeling  and  Simulation. 

First,  “The  purpose  of  the  LFT&E  Management  Plan  is  not  clear.”  [Ref.  103]  The 
IDA  representative  specifically  states,  “The  draft  Management  Plan  contains  a  number  of 
susceptibility  issues,  which  are  not  usually  included  in  LFT&E.  Also,  there  is  probably 
excessive  detail  about  specification  compliance  testing,  normally  addressed  specifically 
as  part  of  developmental  testing  (DT).”  [Ref.  103]  It  is  this  “excessive  detail  on 
irrelevant  subjects”,  and  “insufficient  detail  on  required  subjects”  that  raises  the  concern 
about  the  LFT&E  Management  Plan’s  purpose.  [Ref.  103] 

Second,  the  IDA  representative  links  the  first  concern  with  specific  details. 
Sufficient  detail  in  the  areas  of  schedule,  resources,  component  and  subsystem  testing, 
and  threat  shot  selection  procedures  for  FUSL  are  considered  “clearly  lacking.”  [Ref. 
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103]  Additionally,  susceptibility  and  specification  compliance  testing  is  considered  “not 
directly  relevant  to  the  LFT&E  effort.”  [Ref.  103] 

Finally,  the  IDA  representative  concludes  that  the  LFT&E  Management  Plan 
contains  too  many  different  references  to  M&S  throughout  the  entire  document  without 
sufficient  detail.  The  recommendation  is  that,  “References  to  M&S  should  be 
consolidated  into  a  single  section  devoted  to  the  subject.”  [Ref.  103]  The 
recommendation  specifies  that  the  M&S  section  should  contain  the  following  [Ref.  103]: 

•  An  outline  of  the  overall  application  of  M&S. 

•  How  M&S  supports  the  vulnerability  or  lethality  evaluation. 

•  What  specific  models  are  to  be  proposed. 

•  What  organization  operates  and  supports  the  models. 

•  What  the  model’s  capabilities  and  limitations  are. 

•  What  the  Verification,  Validation,  and  Accreditation  (VV&A)  strategy  is  for 
each  model. 

Part  of  DOT&E’s  concern  over  the  LFT&E  Management  Plan’s  content  related  to 
whether  the  document  would  or  would  not  be  on  the  formal  document  approval  list. 
DOT&E  felt  that  the  LFT&E  Management  Plan  should  be  a  formally  approved  document 
if  the  TEMP’s  description  of  Live  Fire  remained  unchanged.  This  belief  was  based  on 
the  perception  that  the  TEMP  would  contain  only  an  LFT&E  overview  and  that  the 
LFT&E  Management  Plan  would  contain  the  specifics.  In  such  case,  the  Management 
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Plan  would  need  to  incorporate  the  specific  detailed  comments  made  by  both  IDA  and 
DOT&E.  [Ref.  104] 

The  PMO  saw  no  advantage  to  making  the  LFT&E  Management  Plan  a  formal 
document  requiring  DOT&E  signature.  They  saw  the  disadvantage  as  reducing  the 
PMO’s  ability  to  make  changes  to  the  LFT&E  Management  Plan,  and  increasing  the 
“stovepipe”  effect  on  the  planning  process.  The  PMO  felt  the  LFT&E  Management  Plan 
was  simply  a  tool  for  the  LFT&E  Working  Group,  and  because  DOT&E  had  membership 
on  the  Working  Group  that  formal  approval  was  not  necessary.  [Ref.  1 04] 

The  PMO  believed  that  the  advantage  to  keeping  the  LFT&E  Management  Plan 
an  informal  working  document  was  that  minor  and  near  term  changes  could  be  made  by 
the  T&E  IPT  without  processing  through  the  formal  documentation  process.  The 
disadvantage  to  keeping  the  LFT&E  Management  Plan  informal  was  that  the  TEMP’s 
LFT&E  section  would  need  significant  improvement  in  scope  and  detail.  [Ref.  104] 

The  T&E  IPT  ultimately  determined  that  the  TEMP’s  LFT&E  content  would  be 
improved,  the  LFT&E  Management  Plan  would  remain  an  informal  working  document 
for  the  LFT&E  Working  Group,  and  the  Detailed  Test  Plan  for  LF  testing  would  be  a 
formal  document  requiring  DOT&E  signature.  [Ref.  104] 

Lesson:  Coordination  between  the  PMO  and  DOT&E(LFT)  is  critical  for  the 
development  of  a  Live  Fire  Test  and  Evaluation  Management  Plan.  Additionally,  the 
PMO  needs  to  have  a  clear  understanding  of  its  LFT&E  objectives  prior  to  determining 
the  most  appropriate  management  plan  method. 
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F.  REQUIREMENTS 

Changes  in  requirements  for  the  AAAV(C)  by  MCCDC  and  a  recent  mandate  for 
the  inclusion  of  an  Interoperability  KPP  by  the  Joint  Chiefs  of  Staff  have  presented  the 
DRPM  with  two  issues. 

1.  AAAV(C)  Requirements 

The  release  of  the  AAAV(C)  Concept  of  Employment  by  MCCDC  in  January 
1997  led  to  the  DRPM’s  decision  to  stop  work  on  General  Dynamic’s  development  of  the 
AAAV(C). 

Issue:  How  does  the  DRPM’s  “stop  work”  decision  on  the  AAAV(C)  due  to 
changes  in  the  system ’s  requirements  affect  the  test  strategy? 

Discussion:  The  original  test  concept  called  for  the  AAAV(P)  and  the  AAAV(C) 
to  undergo  testing  on  a  parallel  track  throughout  DT-I,  DT-II,  the  various  EOAs,  and  into 
IOT&E.  However,  the  “stop  work”  decision  forced  the  testing  of  the  AAAV(C)  to  occur 
in  an  off-cycle  from  the  AAAV(P).  The  repercussions  were  that  scheduled  EOAs  where 
either  cancelled  in  their  entirety,  or  restructured  and  shifted  to  a  later  period  in  the 
schedule.  Additionally,  Operational  Assessments  were  added  to  partially  compensate  for 
the  loss  of  AAAV(C)  related  EOAs  (See  Appendix  D  and  E). 

The  DRPM’s  Test  Manager  identified  the  disadvantage  of  offset  testing  of  the 
AAAV(C)  and  AAAV(P)  as  that  test  planning  and  conduct  efforts  now  requires 
duplication  of  effort  on  the  part  of  the  DRPM  and  MCOTEA.  Also,  OSD  oversight 
managers  now  view  the  separated  test  events  as  adding  to  the  complexity  of  the 
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program’s  test  strategy,  and  thereby  increasing  the  associated  risk  of  not  successfully 
meeting  cost,  schedule,  and  performance  requirements.  [Ref.  93]  The  DOT&E 
representative  even  stated  that  he  believed  the  development  and  successful  testing  of  the 
AAAV(C)  variant,  “was  the  high-risk  area  to  the  program  and  the  Marine  Corps’  entire 
OMFTS  concept.”  [Ref.  96] 

Advantages  to  the  offset  testing  of  the  two  variants  include  a  more  stable  and  even 
distribution  of  test  personnel  requirements  over  the  entire  test  period.  Instead  of  surging 
test  personnel  for  one  two-variant  DT-I  evolution,  the  DRPM  and  GDAMS  can  maintain 
a  reduced  number  of  test  personnel  spread  over  two  one-variant  DT-I  evolutions. 
Additionally,  the  two  separate  test  events  allow  for  a  better  isolation  of  system 
malfunctions  during  testing.  By  waiting  approximately  12  months  to  conduct  the 
AAAV(C)  DT-I,  the  basic  AAAV  system  design  will  be  more  mature.  This  maturity  will 
allow  testers  and  engineers  to  better  isolate  faults  caused  by  the  newly  integrated 
communications  suite  vice  faults  occurring  in  the  basic  system.  [Ref.  93] 

DOT&E  expressed  two  primary  concerns  after  reviewing  the  DRPM’s  offset  test 
plan  in  November  1998.  First,  the  Operational  Assessments  did  not  appear  to  support 
specific  program  or  technical  decision  points.  DOT&E’s  guidance  to  correct  this 
problem  was  that,  “The  testing  schedule  should  be  revised  to  clearly  show  the  linkage  to, 
and  support  of,  major  program  or  technical  milestones  (such  as  Critical  Design  Review).” 
[Ref.  95]  DOT&E  also  recommended  improving  the  linkage  to  program  milestone 
decisions  or  the  related  technical  engineering  milestones  which  would  use  the  test  results 


190 


to  influence  design.  This  leads  to  DOT&E’s  second  concern  that  the  compressed  time  to 
produce  an  AAAV(C)  mockup  and  the  EMD  design  effort,  “...increases  the  risk  of 
influencing  redesign  vice  design...”  of  the  vehicle.  [Ref.  96] 

DOT&E  also  accepted  the  PMO’s  view  that  IOT&E  should  focus  on  the 
AAAV(C)’s  unique  command  aspects.  The  remainder  of  the  AAAV(C)  characteristics 
can  then  be  estimated  by  regression  testing  from  AAAV(P)  characteristics  in  order  to 
reduce  risk  on  those  characteristics.  [Ref.  96] 

Lesson:  The  affect  on  a  PM’s  test  strategy  based  on  requirement  changes  to  the 
primary  system  could  be  significant,  and  would  generally  require  a  detailed  analysis  of 
each  cases  circumstances.  The  changing  requirements  for  the  AAAV(C)  variant  did  not 
significantly  affect  test  strategy  development.  Impact  was  minimized  because  the 
AAAV(C)  is  a  Mission  Role  Variant  of  the  primary  system  to  which  there  were  no 
requirement  changes.  In  all  cases,  requirement  changes  should  be  kept  to  a  minimum 
when  ever  possible. 

2.  Interoperability  KPP 

The  recently  released  CJCSI  3170.01A  Requirements  Generation  System,  10 
August  1999,  mandates  in  Enclosure  E  that,  “Information  Exchange  Requirements  (IERs) 
are  to  be  used  as  the  primary  basis  and  measure  for  system  interoperability  in  defining 
Interoperability  KPP  threshold  (T)  and  objective  (0)  requirements  for  ORDs...”  [Ref.  95] 
CJCSI  31 70.0 1A  further  outlines  the  connection  between  the  Joint  Pub  1-02  definition  of 
interoperability  and  the  interoperability  KPP  requirement  as  follows: 
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Joint  Pub  1  -02  definition  (2)  for  interoperability  defines  it  as  the  condition 
achieved  among  communications-electronics  systems  or  items  of 
communications-electronics  equipment  when  information  or  services  can 
be  exchanged  directly  and  satisfactorily  between  them  and/or  their  users. 
Even  though  there  are  many  facets  of  interoperability  (e.g.,  fuel, 
ammunition,  transportation,  communications)  that  need  to  be  identified  in 
the  ORD  the  focus  for  the  interoperability  ORD  KPP  will  be  the 
information  exchange  and  interoperability  level  for  the  ORD  system 
information  needs.  The  intent  is  for  the  warfighter  to  outline  the  essential 
information  exchange  requirements  for  the  system  as  described  above. 
The  requirements  should  reflect  both  the  information  needs  necessary  to 
satisfy  the  system  under  consideration  and  the  information  this  new 
capability  can  provide  to  enhance  fielded  systems.  The  development  of 
the  information  exchange  requirements  should  cover  both  the 
communication  requirements  for  command  and  control  of  the  proposed 
system  and  the  level  of  integration  for  cross  system  operations...  ORDs 
will  have  an  Interoperability  KPP.  [Ref.  95] 


Issue:  How  does  the  DRPMbest  address  the  interoperability  KPP  requirement  in 
the  Test  and  Evaluation  Master  Plan? 

Discussion:  The  AAAV’s  current  interoperability  requirements  are  outlined  in 
the  Critical  Technical  Parameters  listed  in  the  1995  TEMP.  However,  the  DRPM’s  Test 
Manager  indicated  that  JROC  desires  the  required  Interoperability  KPP  to  be  listed  as  a 
Critical  Operational  Issue  with  corresponding  Measures  of  Effectiveness  and  Measures  of 
Performance.  The  DRPM’s  Test  Manager  initially  added  interoperability  as  an  MOP  to 
the  MOE  and  MOP  Testable  Criteria  Relationships  Matrix  in  the  1999  Draft  TEMP  (See 
Table  4,  Chapter  5).  The  interoperability  MOPs  where  then  deleted  from  the  Draft  TEMP 
because  of  a  lack  of  guidance  on  exactly  how  to  measure  and  quantify  interoperability 
with  respect  to  communications-electronics  equipment.  [Ref.  93] 
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The  PMO’s  specific  concern  is  how  to  quantify  the  acceptable  levels  of 
degradation  in  the  AAAV(C)’s  capability  verses  a  stationary  or  better-equipped 
Command  Post  configuration.  Other  concerns  about  the  interoperability  KPP 
requirement  is  the  lack  of  distinction  between  addressing  interoperability  in  the  CTPs  and 
addressing  interoperability  in  the  COIs.  The  Test  Manager  believes  that  the  revised  COIs 
already  address  interoperability  with  specific  references  to  “...an  AAAV-equipped 
amphibious  force  supporting]  the  Navy-Marine  Corps  Team’s  concept  of  an  Over-The- 
Horizon  (OTH)  amphibious  assault...”  The  issue  remains  how  best  then  to  address 
interoperability  as  an  MOE  and  MOP  without  further  guidance  from  either  MCCDC  or 
JCS. 

The  present  resolution  to  the  interoperability  issue  is  to  include  a  description  of 
Joint  Interoperability  Test  Command  (JITC)  involvement  in  interoperability  KPP 
development  in  the  Draft  TEMP’s  Part  III  -  Developmental  Test  and  Evaluation  Outline. 
Additionally,  the  current  references  to  interoperability  in  the  CTPs  will  be  deleted  from 
the  final  version  of  the  1999  TEMP  (See  Appendix  I).  Of  the  five  interoperability  CTPs 
listed,  four  did  not  specifically  relate  to  interoperability  as  defined  and  used  in  CJCSI 
3 170.01  A.  The  PMO  continues  to  work  with  MCCDC,  MCOTEA,  and  JITC  to  refine 
interoperability  MOEs  and  MOPs  for  inclusion  in  future  changes  to  the  TEMP.  [Ref.  93] 

Lesson:  The  requirement  to  include  an  interoperability  KPP  in  the  ORD  has 
created  confusion  at  the  Test  Manager  level.  Translation  of  the  requirement  into  MOEs 
and  MOPs  for  development  and  incorporation  into  specific  test  events  is  proving  to  be 
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difficult.  Planned  coordination  with  JITC  may  result  in  a  better  understanding  of  the 
interoperability  KPP  requirement,  and  its  subsequent  translation  into  MOEs  and  MOPs. 

3.  Other  Requirements  Lessons  Learned 

Lesson:  It  is  essential  that  the  initial  Operational  Requirements  Document  (ORD) 
be  drafted  thoroughly,  and  be  written  in  such  a  manner  that  it  allows  for  testability.  A 
well-prepared  and  thorough  ORD  establishes  clarity  for  the  user,  the  Program  Manager, 
the  Contractor,  the  Test  Managers,  and  the  Testing  Agencies.  In  order  for  the  ORD  to 
achieve  this  level  of  thoroughness,  all  interested  parties  need  to  be  involved  in  its 
development  through  the  IPT  process  as  early  as  possible.  Failure  to  do  so  creates 
confusion  among  the  interested  parties,  results  in  conflicting  perceptions  as  to  the 
system’s  true  requirements,  and  prevents  optimization  of  the  test  schedule.  [Ref.  94] 

G.  OTHER  LESSONS  LEARNED 

Lesson:  Cost  and  schedule  are  two  parameters  that  should  be  analyzed 
extensively  for  risk.  Adding  funding  to  the  program  to  allow  building  of  additional 
prototypes  or  increasing  test  resources  can  aid  in  mitigating  schedule  risk.  However,  the 
program  will  reach  a  point  when  schedule  risk  can  no  longer  be  countered  with  funding 
increases.  It  is  important  to  assess  ahead  of  time  when  that  culminating  point  will  occur, 
and  have  contingencies  for  mitigating  increased  risk  without  added  funding.  With  respect 
to  cost  risk,  maintaining  a  stable  test  budget  significantly  aids  in  achieving  and 
maintaining  acceptable  levels  of  risk.  A  cut  to  the  test  budget  in  order  to  apply  funds  to 
other  program  areas  is  a  sure  means  of  escalating  risk  to  an  unacceptable  level.  [Ref.  93] 
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Lesson:  Do  not  overlook  the  importance  of  Environmental  Impact  issues  on  test 
planning.  Areas  to  consider  during  test  planning  include  animal  migratory  patterns  and 
periods,  designated  species’  breeding  seasons,  and  seasonal  human  activities  such  as  peak 
boating  or  fishing  periods.  Test  planners  must  assess  the  impact  of  these  type  activities 
on  both  the  primary  test  schedule  and  alternate  test  schedules.  These  activities  could 
even  result  in  the  elimination  of  certain  test  objectives  because  of  the  increased  costs  to 
delay  testing  until  an  acceptable  period.  Information  about  these  issues  must  be  sought 
by  the  program  management  office,  and  coordinated  with  the  test-site  and  base  range 
control  personnel.  As  an  example,  the  AAAV  PMO  spent  $16,000  on  an  Environmental 
Impact  assessment  for  the  planned  open-ocean  high  water  speed  test  events,  yet  the  Ft. 
Story  tests  were  cancelled  because  of  schedule  slippage.  However,  had  the 
Environmental  Impact  assessment  not  been  conducted,  the  program  would  not  have  been 
able  to  plan  any  testing  at  Ft.  Story.  Environmental  Impact  assessments  and  requirements 
must  be  reviewed  and  planned  for  in  a  “worst  case”  scenario  in  order  to  mitigate  its 
impact  on  test  planning  and  schedule  risk.  [Ref.  93] 

Lesson:  Field  support  is  critical  when  test  personnel  are  away  from  the  contractor 
facility  and  in  the  field  actually  testing  the  system.  Too  often  the  higher  level  supervisors 
have  an  “out  of  sight,  out  of  mind”  mentality.  This  causes  delays  with  the  continuation  of 
testing  when  the  system  needs  corrective  maintenance.  One  solution  to  meet  this 
concern,  is  the  contractor’s  development  and  use  of  a  “virtual  co-location”  system. 
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Television  and  Computer  up-links  will  be  used  to  keep  higher  supervisory  personnel  in 
real  time  or  near-real  time  contact  with  test  personnel  in  the  field.  Virtual  co-location 
will  also  allow  engineers  at  the  facility  to  actually  observe  when  and  how  the  system 
performed  or  failed  to  perform.  Faster  electronic  data  transformation  and  video/TV 
viewing  of  an  actual  malfunction  allows  engineers  to  identify  both  short-term  and  long¬ 
term  solutions  for  corrective  actions  in  order  to  get  the  system  returned  to  operating 
condition.  Generally,  long-term  solutions  are  much  different  than  short-term  solutions 
and  the  contractor  beliefs  that  “The  sooner  action  is  taken,  the  sooner  design  corrections 
can  be  implemented  for  continuation  of  future  testing  and  improvement  of  system 
design.”  [Ref.  92] 

Lesson:  Having  experienced  Assault  Amphibian  Officers  as  MCOTEA’s  AAAV 
Operational  Test  Director,  MCCDC’s  AAAV  Requirements  Action  Officer,  AVTB’s 
Director  and  Logistics  Officer,  and  throughout  the  entire  DRPM  Office  results  in  a  clear 
understanding  by  all  elements  of  the  acquisition  process  as  to  the  needs  of  the  user.  This 
common  understanding  also  serves  to  reduce  confusion  between  the  separate  agencies, 
allows  for  faster  decision  making,  and  may  ultimately  result  in  a  better  designed  system 
that  meets  user  requirements. 

F.  CHAPTER  SUMMARY 

This  chapter  analyzed  and  discussed  those  issues  facing  the  AAAV  Program 
Management  Office  that  had,  or  potentially  have,  an  impact  on  the  development  and 
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conduct  of  the  program’s  test  and  evaluation  strategy.  Where  available,  the  courses  of 
action  (COAs)  available  to  the  PMO,  and  the  associated  advantages  and  disadvantages  of 
the  COAs,  were  included.  Applicable  lessons  learned  were  then  presented  for  each  issue 
in  order  to  provide  potential  guidance  for  other  program’s  facing  similar  issues. 
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VII.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  INTRODUCTION 

The  objective  of  this  thesis  was  to  examine  the  evolution  of  the  AAAV  Program 
Management  Office’s  test  and  evaluation  strategy  from  Milestone  0  to  the  present.  The 
goals  were:  (1)  to  analyze  test  and  evaluation  issues  facing  the  Program  Management 
Office,  (2)  determine  the  effects  of  the  issues  on  the  program’s  test  strategy,  and  (3) 
develop  applicable  lessons  for  other  major  defense  acquisition  programs. 

Research  involved  reviewing  the  evolution  of  amphibious  doctrine  and  the 
subsequent  evolution  of  amphibious  vehicles  to  support  that  doctrine,  reviewing  the  DoD 
Acquisition  Process  and  the  role  of  Test  and  Evaluation  in  that  Acquisition  Process,  and 
then  analyzing  three  DRPM-AAA  Test  and  Evaluation  Master  Plans. 

General  and  specific  conclusions  are  provided  about  the  evolution  of  the  DRPM’s 
test  and  evaluation  strategy  and  the  issues  that  effected  its  development  and  execution. 
Specific  recommendations  are  provided  for  other  Major  Defense  Acquisition  Programs, 
and  recommended  future  research  questions  are  proposed. 

B.  CONCLUSIONS 

1.  General  Conclusions 

Test  and  Evaluation  Strategy:  The  Test  and  Evaluation  Strategy  for  the  DRPM- 
AAA  remained  stable  and  consistent  throughout  the  eleven-year  period  from  Milestone  0 
to  the  present.  This  strategy  stability  is  attributed  to  the  continuity  of  the  AAAV’s  Key 
Performance  Parameters  (KPPs).  The  desired  performance  characteristics  for  an 
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amphibious  vehicle  began  even  prior  to  the  establishment  of  the  AAAV  program  office, 
and  have  remained  essentially  unchanged  for  nearly  twenty  years.  Because  of  this, 
program  office  test  personnel  were  able  to  establish  clear  and  concise  goals  for  the 
conduct  of  the  program’s  test  efforts. 

Lesson  Applicability:  The  lessons  learned  from  analysis  of  the  DRPM-AAA’s 
Test  and  Evaluation  Strategy  are  applicable  to  most  major  defense  acquisition  programs 
and  should  be  applied  accordingly. 

2.  Specific  Conclusions 

Developmental  Test  Strategy:  The  DRPM’s  Developmental  Test  Strategy  has 
remained  consistent  throughout  the  program.  The  DT  strategy  is:  To  confirm  the 
AAAV’s  technical  performance  characteristics,  to  determine  the  feasibility  of  selected 
system/sub-system  or  component  technical  design,  to  determine  the  degree  of  risks,  and 
to  initially  assess  logistical  supportability,  reliability,  and  maintainability  of  the  system. 
This  has  allowed  the  program  to  maintain  its  developmental  testing  focus  without 
becoming  distracted  by  higher  level  issues. 

Operational  Test  Strategy:  The  DRPM’s  Operational  Test  Strategy  has 
remained  consistant  throughout  the  program.  The  OT  strategy  is:  To  determine  if  the 
AAAV  meets  minimum  operational  effectiveness  and  operational  suitability  requirements 
appearing  in  the  approved  ORD  by  evaluating  the  AAAV’s  performance  during  EOAs, 
OAs,  IOT&E,  LFT&E,  and  selected  DT  events  in  order  to  support  acquisition  decisions. 
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Despite  changes  in  the  program’s  funding,  schedule,  and  AAAV(C)  requirements,  the 
stable  OT  strategy  has  given  the  program  the  flexibility  to  modify  and  cancel  EOAs,  add 
OAs,  and  refine  IOT&E  events  in  order  to  adapt  to  a  changing  environment. 

Acquisition  Reform:  The  acquisition  reform  efforts  of  down  selection  to  one 
contractor  in  the  PDRR  phase,  co-location  of  the  Government  and  the  Contractor  in  the 
same  facility,  and  implementation  of  the  IPPD/ITP  process  have  proven  invaluable  to  the 
test  planning  process.  The  first  two  reform  measures  are  linked  directly  to  the  latter,  and 
have  resulted  in  a  synergistic  effect  for  the  program.  Without  these  combined  reform 
measures  the  program  would  not  have  been  successful  in  planning,  coordinating,  and 
conducting  other  reform  measures  that  meet  the  DRPM’s  optimization  goal.  Integrated 
Contractor  and  Government  developmental  testing  could  not  have  been  coordinated, 
Detailed  Test  Plans  written  by  the  Contractor  would  not  have  been  acceptable  to  the  test- 
site  agencies,  and  generation  of  the  Test  and  Evaluation  Management  Plan  and  the  Live 
Fire  Test  and  Evaluation  Management  Plan  would  have  been  unacceptably  delayed 
without  these  acquisition  reform  measures. 

Working  Relationships:  The  DRPM’s  decision  to  develop  and  maintain  a 
working  relationship  that  “actively  engages”  both  the  oversight  agencies  and  the  external 
agencies  early  in  the  test  planning  process  served  three  functions.  First,  the  inclusiveness 
mind-set  allowed  the  program  office  to  accomplish  its  mission  while  still  meeting  the 
other  agency’s  requirements.  Second,  by  using  the  agencies’  knowledge  and  expertise, 
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the  program  is  achieving  a  greater  efficiency  in  its  use  of  test  resources.  Finally,  both 
sides  can  continue  to  reach  balanced  and  acceptable  levels  of  testing  within  the  realities 
of  the  program’s  fiscal  constraints. 

Test  Manager:  The  congressional  funding  plus-ups,  the  increased  procurement  of 
PDRR  prototypes,  and  the  schedule  acceleration  concessions  were  out  of  the  test 
manager’s  control,  yet  each  had  a  significant  impact  on  his  ability  to  plan,  coordinate,  and 
prepare  for  the  conduct  of  the  AAAV’s  test  program.  The  DRPM’s  test  manager  was 
able  to  overcome  these  issues  because  of  his  understanding  of  the  DRPM’s  test  strategy 
and  his  relationship  with  other  agencies.  The  combination  of  a  clear  test  strategy  and 
good  working  relationships  placed  the  test  manager  in  a  better  position  to  maintain  an 
awareness  of  the  program’s  overall  test  status,  make  necessary  changes  to  the  test  plan, 
manage  the  test  program,  and  achieve  system  test  objectives. 

Optimization:  The  DRPM  has  made  every  effort  to  optimize  Government  and 
Contractor  time  and  resources  to  ensure  an  efficient  and  effective  test  program.  This 
optimization  mind-set  was  established  by  the  DRPM  in  conjunction  with  Acquisition 
Reform  efforts,  and  has  resulted  in  the  program  continually  assessing  the  system’s 
requirements  and  the  test  objectives  to  ensure  that  redundancies  are  justified  or 
eliminated.  True  optimization  would  not  be  possible  if  oversight  and  external  agencies 
had  not  been  “actively  engaged”  early  in  the  process. 
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T&E  WIPT:  The  Test  and  Evaluation  Working-Level  IPT  is  proving  essential  for 
the  successful  implementation  of  the  program’s  Test  and  Evaluation  Strategy.  The  T&E 
WIPT  is  a  “value  added”  management  tool  that  enables  enhanced  communication, 
ensures  mutual  understanding,  and  prevents  confusion  among  the  critical  decision  makers 
at  the  appropriate  level.  The  DRPM’s  development  of  a  subordinate  LFT&E  Working 
Group  has  proven  successful  in  meeting  OSD  oversight  requirements.  Considerations  to 
implement  similar  subordinate  Working  Groups  for  Developmental  Testing  and 
Operational  Testing  are  situationally  dependent.  As  such,  each  Program  Management 
Office  must  determine  the  most  appropriate  Test  and  Evaluation  management  techniques 
based  on  that  program’s  specific  needs. 

Virtual  Co-location:  The  contractor’s  development  and  planned  use  of  a  “virtual 
co-location”  system  to  keep  supervisory  personnel  and  engineers  in  real-time  or  near  real¬ 
time  contact  with  test  personnel  in  the  field  appears  to  enhance  test  resource  optimization. 
Enabling  engineers  at  the  facility  to  observe  how  the  system  performs  or  fails  to  perform, 
increasing  electronic  data  transformation,  and  having  video  recordings  of  actual 
malfunctions  should  enable  engineers  to  identify  short-term  and  long-term  solutions  in  a 
more  effective  and  efficient  manner.  Faster  and  more  effective  solutions  should  reduce 
the  schedule  risk  associated  with  system  malfunctions  and  test  delays. 

IPT  Decision-Making:  Those  military  members  assigned  to  DRPM-AAA 
without  previous  acquisition  experience  had  not  been  exposed  to  the  IPPD/IPT  process, 
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and  therefore  did  not  initially  understand  the  difference  between  the  IPT  decision-making 
process  and  the  more  traditional  “staff  planning  process.” 

TEMP  Update:  The  PMO  has  been  successful  in  generating  an  updated  TEMP 
because  of  four  key  decisions.  First,  the  program  convened  the  process  as  early  as 
possible.  Second,  the  program  actively  sought  input  from  all  OSD  oversight  agencies 
especially  with  respect  to  Live  Fire  Test  and  Evaluation.  Third,  the  program  worked 
closely  with  the  requirements  developer  to  ensure  that  the  ORD  update  process  paralleled 
the  TEMP  update  process  ensuring  continuity  between  the  two  documents.  Finally,  the 
program  maintained  close  and  constant  coordination  with  the  independent  operational  test 
director  from  MCOTEA.  This  coordination  allowed  MCOTEA  and  the  DRPM  to 
develop  Part  IV  of  the  TEMP  with  a  significant  level  of  continuity. 

COIs:  Though  the  new  COIs  address  the  use  of  the  AAAV  in  an  amphibious 
force  supporting  both  the  Navy-Marine  Corps  Team  concept  and  the  OMFTS  OTH 
concept,  the  COIs  do  not  reference  the  use  of  the  AAAV  in  support  of  a  Marine  Air 
Ground  Task  Force  (MAGTF).  Inclusion  of  the  term  MAGTF  in  any  one  of  the  COIs 
would  increase  the  level  of  assumed  interoperability  and  compatibility  necessary  for  the 
AAAV  system  in  an  operational  environment.  Inclusion  of  the  MAGTF  concept  would 
also  increase  the  level  of  understanding  of  the  AAAV’s  required  capabilities  by  those 
unfamiliar  with  the  AAAV,  but  who  are  familiar  with  MAGTF  operations. 
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C.  RECOMMENDATIONS 


1.  Program  Managers  and  Requirements  Representatives  should  determine  a 
system’s  Key  Performance  Parameters  (KPPs)  as  early  in  the  acquisition  process  as 
possible,  and  then  aggressively  defend  those  KPPs  against  “better”  requirements.  Stable 
and  consistent  KPPs  establish  the  foundation  for  the  program  and  its  entire  test  effort. 

2.  Program  Managers  and  Program  Test  Managers  should  review  the  lessons 
learned  for  applicability  to  other  major  defense  acquisition  programs. 

3.  Program  Manager’s  should  give  full  consideration  to  the  benefits  of  the 
acquisition  reform  efforts  of  down  selection  to  one  contractor  in  the  PDRR  phase,  co- 
location  of  the  Government  and  the  Contractor  in  the  same  facility,  and  implementation 
of  the  IPPD/ITP  process. 

4.  Program  Managers  should  emulate  the  DRPM- AAA’s  decision  to  develop  and 
maintain  a  working  relationship  that  “actively  engages”  both  the  oversight  agencies  and 
the  external  agencies  early  in  the  test  planning  process  in  order  to  best  meet  agency 
requirements,  exploit  agency  knowledge  and  expertise,  and  achieve  enhanced  test 
resource  efficiency. 

5.  Program  Managers  considering  a  co-location  decision  should  account  for  the 
segmenting  of  the  contractor’s -test  capabilities  by  assessing  the  number,  experience,  and 
skill  level  of  test  personnel  required  for  projects  based  on  the  level  of  system  complexity. 
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6.  Military  members  newly  assigned  to  a  program  office  should  be  trained  to 
understand  the  differences  between  the  IPT  decision-making  process  and  the  more 
traditional  “staff  planning  process.” 

7.  The  term  MAGTF  should  be  included  in  at  least  one  of  the  AAAV’s  COIs  in 
order  to  increase  the  level  of  assumed  interoperability  and  compatibility  necessary  for  the 
AAAV  system  in  an  operational  environment,  and  to  increase  the  level  of  understanding 
of  the  AAAV’s  required  capabilities  by  those  familiar  with  MAGTF  operations. 

D.  AREAS  FOR  FURTHER  RESEARCH 

As  a  result  of  this  research  effort,  the  author  recommends  the  following  test  and 
evaluation  questions  for  further  research: 

•  What  role  can,  or  does,  Modeling  and  Simulation  play  in  reducing 
Developmental  Testing,  Operational  Testing,  and  Live  Fire  Test  and 
Evaluation? 

•  Is  it  justifiable  to  use  EMD  prototypes  in  place  of  LRIP  articles  for  IOT&E, 
and  then  use  LRIP  articles  for  FOT&E? 

•  What  are  the  guiding  principles  for  integrating  Developmental  Testing  and 
Operational  Testing? 

•  How  can  the  effectiveness  and  suitability  of  system  software  best  be  tested? 

•  How  effective  is  a  Model-Test-Model  approach  to  system  development? 
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APPENDIX  A.  DIRECT  REPORTING  PROGRAM  MANAGER,  ADVANCED 
AMPHIBIOUS  ASSAULT  PROGRAM  ORGANIZATION 
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APPENDIX  B.  AAAV  PROGRAM  LEVEL  INTEGRATED  PRODUCT  TEAMS 

(IPT) 


Modeling  &  Simulation  Team 
Logistics  Team 

Speciality  Engr.  Team 

ElectfPwr  Mgmt.  Team 


Controls  &  Displays 
Team 


Comm/Nav  Team 


Software  Team 


Team 


Auxiliai 

System 

Team 


Engine  Team 


Hydro  Syst  & 
Appendages  Team 


Marine  Drivetrain  Team 
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APPENDIX  C.  TEST  AND  EVALUATION  RESPONSIBILITIES 


The  following  is  a  list  of  test  and  evaluation  responsibilities  as  delineated  in  the  DRPM 
AAA’s  1999  Draft  Test  and  Evaluation  Master  Plan  [Ref.  57:pp.  II-2  through  II- 7]: 


1.  DIRECT  REPORTING  PROGRAM  MANAGER  AAA. 

a.  Coordinate  all  test  and  evaluation  planning  in  the  TEMP. 

b.  Plan,  program  and  formulate  budgets  associated  with  AAAV  test  and 
evaluation.  Coordinate  completion  of  all  test  products  required  for  I-IPT,  O-IPT,  and 
Milestone  Reviews. 

c.  Update  TEMP. 

d.  Chair  the  T&EWIPT. 

e.  Issues  the  Safety  Assessment  for  all  phases  of  testing,  including  presentations 
to  the  Weapon  System  Explosive  Safety  Review  Board  (WSESRB). 

f.  Chair  RAM  Scoring  Conference  for  Developmental  Testing.  The  Failure 
Definition  and  Scoring  Criteria  (FDSC)  utilized  for  the  AAAV  test  program  will  be 
adjudicated  by  MCOTEA,  DRPM  AAA  and  MCCDC  prior  to  the  beginning  of  testing. 

g.  Develop  test  issues  based  on  thresholds  established  in  the  AAAV  Operational 
Requirements  Document. 

h.  Establish  and  execute  the  Developmental  Test  program  for  the  AAAV. 

(1)  Identify  and  establish  responsibilities  for  test  facilities.  Identify  resources 
required  to  accomplish  DT. 

(2)  Coordinate  test  schedules,  test  sites,  instrumentation  and  data  collection 
requirements,  safety  and  environmental  issues,  training  and  logistical  support. 

(3)  Conduct  of  DT  Test  Readiness  Review. 

(4)  Develop  DT&E  test  reports  and  quick-look  reports. 
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(5)  Review  and  approve  Logistics  Demonstration  Plan  developed  by  the 
contractor.  Provide  the  Marine  maintainers  that  will  perform  the  Logistics 
Demonstrati  on(s) . 

i.  DRPM  AAA  will  review  and  analyze  the  transportability  engineering  aspects 
of  the  AAAV. 

j.  Establish,  direct,  fund  and  execute  the  Live  Fire  Test  and  Evaluation  program 
for  the  AAAV.  Significant  management  responsibilities  are  found  in  Part  IV. 

k.  Coordinate  with  MCOTEA  for  the  AAAV  operational  test  program. 

(1)  Establish  program  expectations  for  operational  tests. 

(2)  Participate  in  any  Test  Integration  Working  Group  (TIWG)  established  by 

MCOTEA. 


(3)  Provide  training  for  Marine  operators  and  maintainers  selected  for  all  OT 

events. 

1.  Provide  End-of-Test  Phase  Report  to  DTSE&E  and  DOT&E  listing  the  T&E 
results,  conclusions  and  recommendations  prior  to  Milestone  decision  and  final  decision 
to  proceed  beyond  LRIP. 


2.  MARINE  CORPS  OPERATIONAL  TEST  &  EVALUATION  ACTIVITY 
(MCOTEA). 

a.  Establish  liaison  with  DPRM  AAA  for  test  requirements  and  plans. 

b.  Develop  Operational  Test  plans,  schedules,  and  resources  for  the  TEMP 
Section  IV. 

c.  Participate  on  AAAV  T&E  WIPT. 

d.  Voting  member  of  the  RAM  Scoring  Conference.  Chairs  conference  during 
OT  events. 

e.  Concur  with  the  TEMP. 

f.  Plan  and  conduct  AAAV  OT&E. 
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(1)  Plan  and  chair  Operational  Test  Readiness  Reviews. 

(2)  Coordinate  operational  test  schedules,  sites,  instrumentation  and  data 

collection. 


(3)  Coordinate  with  DPRM  AAA  for  opportunities  to  integrate  DT/OT  events. 

(4)  Coordinate  Fleet  Marine  Force  users  for  AAAV  operational  test  events. 

(5)  Chair  AAAV  Test  Integrated  Working  Group  (TIWG). 

(6)  Support  the  validation  and  accreditation  of  models,  targets  and  threat 

simulators. 

g.  Provide  Operational  Test  Independent  Evaluation  Reports. 

h.  Participate  in  the  AAAV  Live  Fire  Working  Group.  MCOTEA’s 
responsibilities  in  the  context  of  Live  Fire  Test  support  are  delineated  in  Part 
IV. 


3.  MARINE  CORPS  COMBAT  DEVELOPMENT  COMMAND  (MCCDC) 

I  a.  Develop  AAAV  system  requirements  and  operational  doctrine. 

b.  Member  of  the  AAAV  T&E  WIPT. 

c.  Member  of  the  RAM  Scoring  Conference. 

d.  Member  of  the  Live  Fire  Working  Group  and  the  Live  fire  Damage 
Assessment  Team  (DAT).  MCCDC  shall  organize,  train  and  equip  Battle  Damage 
Assessment  and  Repair  (BDAR)  teams  to  perform  BDAR  for  selected  LFT  events. 
MCCDC  will  validate  operational  loadouts  in  support  of  FUSL  testing.  Additionally 
MCCDC  will  provide  for  maintenance  and  repair  support  as  required  for  LFT&E  of  the 
AAAV. 

e.  Concur  with  the  TEMP. 
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4.  MARINE  CORPS  INTELLIGENCE  ACTIVITY  (MCIA) 


a.  Serves  as  a  voting  member  of  the  DAT  and  the  Live  Fire  Working  Group. 
MCIA  provides  threat  assessments  and  definitions  to  the  working  group  to  aid  in  shotline 
selection  for  ballistic  tests  and  to  ensure  that  the  vehicle  is  evaluated  against  realistic 
threats  that  it  will  be  expected  to  see  during  its  service  life. 

b.  Assists  the  DRPM  with  development  of  the  Systems  Threat  Analysis  (STA). 


5.  DIRECTOR  TEST,  SYSTEMS  ENGINEERING,  AND  EVALUATION 
(DTSE&E) 

a.  Approve  the  TEMP. 

b.  Observe  DT&E  to  ensure  adequacy  of  testing  and  to  assess  test  results. 

c.  Provide  technical  assessment  of  DT&E  conducted  on  AAAV. 

d.  Member  of  the  AAAV  T&E  WIPT.  May  participate  in  other  AAAV  IPTs  as 
required. 

e.  Reviews  AAAV  program  documentation  for  DT&E  implication  and  resource 
requirements  to  provide  comments  to  USD(A&T)  and  DAB. 


6.  PISA  JOINT  INTEROPERABILITY  TEST  COMMAND  (JITC) 

a.  Perform  requirements  analysis  of  AAAV  program  documents  and  updates 
(e.g.  MNS,  ORD,  TEMP) 

b.  Support  the  preparation  and  review  of  AAAV  documents. 

c.  Prepare  and  review  detailed  interoperability  test  procedures  for  AAAV 
Command,  Control,  Communications,  Computers  and  Intelligence  (C4I)  systems. 

d.  Attend  interoperability  meetings,  forums  and  conferences. 

e.  Prepare  and  maintain  a  C4I  Interoperability  Certification  Evaluation  Plan 

(ICEP). 

f.  Define  C4I  interoperability  test  data  collection  methods  (DT  and  OT). 
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g.  Provide  test  site(s),  equipment  and  personnel  resources,  as  required,  during 
C4I  interoperability  tests. 

h.  Participate  in  C4I  tests  and/or  review  test  data  as  basis  for  system  certification. 

i.  Provide  system  certification  recommendations  for  AAAV  C4I 
interoperability. 


7.  DIRECTOR  OPERATIONAL  TEST  AND  EVALUATION  (DOT&E)  ' 

a.  Approve  the  TEMP. 

b.  Approve  AAAV  operational  test  plans. 

c.  Observe  T&E  preparation  and  conduct. 

d.  Analyze  results  of  T&E  conducted  on  AAAV. 

e.  Provide  oversight  of  AAAV  Live  Fire  testing  (see  Table  IV-6  for  document 
approval). 

f.  Member  of  the  AAAV  T&E  WIPT. 

g.  Voting  member  of  the  DAT  and  the  Live  Fire  Working  Group. 

h.  Reviews  and  comments  on  the  Service  Vulnerability  Evaluation  Reports. 
Provides  an  independent  vulnerability  report  to  Congress. 

i.  Prepare  AAAV  Beyond  LRIP  report  to  submit  to  SECDEF  and  Congress. 

8.  ARMY  EVALUATION  ANALYSIS  CENTER  (EAC) 

a.  EAC  will  assist  in  the  conduct  and  planning  of  the  Full  Up  System  Live  Fire 
Test  (FUSL).  This  includes  assistance  with  the  development  of  a  detailed  strategy  for  the 
FUSL  test,  assistance  with  evaluation  of  LFT&E  data  and  assistance  with  preparation  of 
the  independent  assessment  of  the  vulnerability  of  the  AAAV. 

b.  Voting  member  of  the  DAT  and  the  Live  Fire  Working  Group. 
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9.  ARMY  RESEARCH  LAB  (ARL) 


a.  Voting  member  of  the  Live  Fire  Working  Group.  Prepares  pretest  predictions 
for  all  Ballistic  Hull  and  Turret  (BH&T)  and  FUSL  events,  collects  data  from  FUSL  test 
events  and  provides  a  damage  assessment  report  detailing  the  results  of  the  FUSL  tests. 

b.  Chairs  the  DAT. 


10.  TEST  SITES 

a.  Aberdeen  Test  Center  (ATC),  Aberdeen  MD 

(1)  Coordinate  land  mobility  test  planning,  resources,  and  execution  for  DT-I. 

(2)  Provide  safety  recommendations  and  DT-I  land  mobility  test  report. 

(3)  Voting  member  of  the  DAT  and  the  Live  Fire  Working  Group.  ATC  will 
prepare  detailed  live  fire  test  plans  and  conduct  testing  as  assigned  by  DRPM  AAA  and 
the  Live  Fire  Working  Group. 


b.  Amphibious  Vehicle  Test  Branch  (AVTB),  Camp  Pendleton  CA 

(1)  Coordinate  water  mobility  test  planning,  resources  and  execution  for  DT-I. 

(2)  Provide  recommendations  and  DT-I  water  mobility  test  report. 

(3)  Support  MCOTEA  in  resource  planning  and  EOA  conduct. 

c.  Eglin  Air  Force  Base 

(1)  Coordinate  firepower  test  planning,  resources  and  execution  for  DT-I. 

(2)  Provide  recommendations  and  DT-I  firepower  test  report. 

d.  Test  agencies  and  test  site  locations  for  EMD  (DT-II)  have  yet  to  be 
determined. 
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11.  GENERAL  DYNAMICS  AMPHIBIOUS  MARINE  SYSTEMS  (GDAMSV 

a.  Develop  Detailed  Test  Plans  for  developmental  testing. 

b.  Conduct  acceptance  testing  for  each  prototype. 

c.  Provide  maintenance  and  logistic  support  of  the  prototype  vehicles  (prior  to 
IOT&E). 

d.  Transport  of  the  prototypes  from  test  site  to  test  site. 

e.  Provide  safety  assessment  report  to  support  the  DT  Test  Readiness  Review. 

f.  Develop  operator  and  maintenance  manuals  for  the  AAAV  system  to  support 

testing. 

g.  Provide  operators  and  maintained  for  the  prototype  vehicles. 

h.  GDAMS  will  develop  the  training  curriculum  and  conduct  the  training,  under 
DRPM  AAA  supervision,  for  the  operators  and  maintainers  of  all  test  vehicles. 
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Integrated  Test  Program  Schedule 
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■  Conducted  Test  □  Planned  Test  03  Possible  Test 
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APPENDIX  E.  AAAV  INTEGRATED  TEST  PROGRAM  SCHEDULE  -  1999 
DRAFT  TEST  AND  EVALUATION  MASTER  PLAN 
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*  live  Fire  PDRR  and  EMD  Teels  Include  Component  Ballistic.  Armor  Characterization,  Controlled  Oamage,  AAAV(C)  Controlled  Damage  and  Characterization  Testing 
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APPENDIX  F.  AAAV  LIVE-FIRE  TEST  SCHEDULE  - 1999  DRAFT  TEST  AND 

EVALUATION  MASTER  PLAN 
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1  Some  Component*  an  planned  to  1*  tMtcd  as  purl  of  the  RH&T> 
3  "C"  refen  to  AAAVfC)  variant  unique  testing  and  simulation. 
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APPENDIX  G.  AAAV  COST  AND  OPERATIONAL  EFFECTIVENESS 
ANALYSIS  (COEA)  - 1995  INTEGRATED  PROGRAM  SUMMARY 


Alternatives  Assessed  and  Results 
a.  Alternatives  Considered 

A  total  of  13  basic  alternatives  were  evaluated  during  the  concept  phase.  These 
alternatives  fell  into  four  general  categories:  Low  Speed  Amphibious  Vehicles,  High 
Speed  Amphibious  Vehicles,  Non-Amphibious  Vehicles,  and  Non- Vehicle 
Alternatives. 

Low  Speed  Amphibious  Vehicles  operate  as  displacement  vehicles  when  in  the 
water.  They  are  therefore  limited  to  water  speeds  of  approximately  6-9  knots.  On 
land  these  alternatives  operate  as  tracked  Armored  Personnel  Carriers  (APCs). 
While  able  to  operate  independently  over  short  distances  in  the  water,  they  must  rely 
on  LCACs  for  ship-to-shore  maneuver.  Within  the  category  of  Low  Speed 
Amphibious  Vehicles  are  the  following  specific  alternatives: 

•  AAV7A1 :  The  existing  system. 

•  AAV7A2(S):  A  product  improved  AAV7A  1 . 

•  AAAV(S):  A  newly  designed  low  speed  vehicle. 

•  Submersible:  A  newly  designed  tracked  vehicle  which  would  transit  to  shore 
below  the  surface  of  the  water. 

High  Speed  Amphibious  Vehicles  are  self  deploying  and  operate  independently  from 
LCACs  during  ship-to-shore  maneuver.  They  are  capable  of  water  speeds  far  greater 
than  a  displacement  vehicle.  The  water  speed  of  this  category  of  alternatives  is  20 
knots  or  greater.  High  Speed  Amphibious  Vehicles  operate  as  armored  personnel 
carriers  on  land.  Within  the  category  of  High  Speed  Amphibious  Vehicles  are  the 
following  alternatives: 

•  AAV7A2(F):  A  product  improvement  to  the  existing  AAV7A1  to  attain  high 
water  speed. 

•  AAAV (F):  A  newly  designed  high  water  speed  amphibian  vehicle. 
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Non- Amphibious  Vehicles  are  systems  which  cannot  operate  independently  during 
water  operations  and  must  rely  on  LCACs  for  ship-to-shore  maneuver.  These 
systems  also  operate  as  armored  personnel  carriers  on  land.  Within  the  category  of 
Non- Amphibious  Vehicles  are  the  following  specific  alternatives: 

•  APC(X):  A  newly  designed  armored  personnel  carrier. 

•  LAV-25:  An  existing  wheeled  vehicle  in  the  USMC  inventory. 

•  Ml  13A3:  An  existing  APC  in  the  U.S.  Army  inventory. 

•  M2A2:  An  existing  Infantry  Fighting  Vehicle  in  the  U.S.  Army  inventory. 

•  FIFV :  A  future  infantry  fighting  vehicle  which  was  planned  for  development 
by  the  U.S.  Army. 

Non- Vehicle  Alternatives  are  systems  which  do  not  provide  armor  protected  land 
mobility  or  direct  fire  support  to  Marine  infantry.  These  systems  provide  only  ship- 
to-shore  transportation.  The  Non-Vehicle  alternatives  include  the  following: 

•  All  Surface  Option:  Marine  infantry  carried  ashore  in  temporary  shelters 
aboard  LCACS. 

•  All  Air  Option:  Marine  infantry  carried  ashore  exclusively  by  helicopters. 

b.  Rejected  Alternatives  and  Reasons  for  Their  Non-selection 

The  alternatives  were  analyzed  within  the  initial  COEA  in  three  separate  areas. 
These  areas  were  system  performance,  force  effectiveness,  and  life-cycle  cost.  The 
performance  analysis  examined  the  capabilities  of  each  alternative  relative  to  Ship- 
To-Shore  Movement,  Mobility  Ashore,  Survivability,  and  Lethality.  The  purpose  of 
the  performance  analysis  was  to  screen  the  least  capable  systems  from  the  group  in 
order  to  concentrate  the  bulk  of  analytical  resources  on  the  more  promising 
alternatives.  The  performance  analysis  resulted  in  the  screening  of  six  of  the 
alternatives  from  further  consideration.  These  alternatives  were  the  submersible, 
AAV7A2(F),  M2A2,  FIFV,  All  Air  and  LAV-25.  Both  the  AAV7A1  and  M113A3 
did  extremely  poorly;  however,  the  AAV7A1  was  carried  forth  as  the  baseline  and 
Ml  13 A3  remained  due  to  its  exceptionally  low  cost. 

The  remaining  alternatives  were  then  analyzed  by  looking  at  their  contribution  to 
force  capability  as  a  whole.  Two  force-on-force  scenarios  spanning  the  full  range  of 
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combat  intensity  levels  were  run  for  each  of  the  remaining  alternatives.  The 
effectiveness  analysis  examined  a  range  of  Measures  of  Effectiveness  (MOEs) 
including  Loss  Exchange  Ratio  (LER),  force  movement,  percentage  of  the  force 
surviving,  force  ratio,  and  time  to  move  the  Assault  Element  from  ship  to  shore. 
Three  additional  alternatives  were  screened  as  a  result  of  this  analysis.  These  were 
the  AAV7A1,  M113A3  and  the  Surface  Option.  Finally,  the  four  remaining 
alternatives  were  analyzed  to  determine  life  cycle  costs. 

As  stated  above,  the  performance  analysis  resulted  in  the  screening  of  six  of  the  13 
alternatives  from  further  consideration.  These  were:  the  submersible,  the 
AAV7A2(F),  the  M2A2,  the  FIFV,  the  LAV-25,  and  the  Air  Option.  The 
submersible  was  screened  as  it  was  determined  that  this  alternative  offered  no 
appreciable  performance  improvements,  but  included  a  significant  amount  of 
technical  and  operational  risk  associated  with  this  new  technology.  Due  to  the 
restrictions  imposed  by  the  vehicle's  existing  hull  form  the  AAV7A2(F)  was  found  to 
be  a  high  risk  technical  venture  and  would  require  nearly  twice  as  many  vehicles  to 
carry  the  same  number  of  personnel  due  to  its  reduced  capacity  caused  by  the 
integration  of  high  water  speed  machinery.  The  M2 A2,  LAV-25  and  FIFV  would 
also  require  a  significant  number  of  increased  vehicles  and  crews  to  carry  an 
embarked  Marine  infantry  unit.  Their  employment  would  also  result  in  the 
requirement  for  additional  amphibious  ships  and  LCACs  to  achieve  minimally 
acceptable  force  mobility  and  force  buildup  rates.  Finally,  the  air  option  was 
screened  in  favor  of  the  surface  option  as  it  provided  no  measurable  difference  when 
considering  the  delivery  of  Marines  to  beach  landing  sites.  This,  combined  with  the 
effect  of  no  potential  increases  in  helicopter  requirements  (or  adverse  effect  of  the 
designated  helicopter  delivered  force)  further  corroborated  the  screening  out  of  the 
Air  Option. 

As  a  result  of  the  operational  effectiveness  analysis,  the  surface  option,  the  Ml  13  A3, 
and  the  AAV7A1  were  all  screened  from  further  consideration.  These  alternatives 
were  found  to  be  significantly  less  capable  than  the  other  remaining  alternatives  in 
both  scenarios  and  for  all  MOEs.  Of  the  13  original  alternatives,  four  remained. 
These  were;  AAV7A2(S),  AAAV(F),  AAAV(S),  and  APC(X). 

c  .  Results  of  Cost  and  Operational  Effectiveness  Analysis 

In  every  measure  of  effectiveness  the  AAAV(F)-equipped  expeditionary  force 
uniformly  surpassed  all  other  alternatives.  This  superiority  also  remained  constant 
throughout  the  sensitivity  analysis  conducted  subsequent  to  the  operational 
effectiveness  analysis.  Given  its  significant  contribution  to  force  effectiveness,  its 
uniform  superiority  to  remaining  alternatives,  and  its  total  compatibility  with  Marine 
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Corps  doctrinal  and  tactical  concepts,  the  AAAV(F)  was  selected  as  the  service 
choice. 

In  1993  an  update  to  the  original  COEA  was  conducted.  This  analysis  evaluated 
several  different  development  and  acquisition  strategies  but  did  not  introduce  new 
system  alternatives.  In  the  updated  COEA  the  baseline  system  is  referred  to  as  the 
AAV7A1E2.  It  is  identical  to  the  original  baseline  system  (AAV7A1)  with  some 
postulated  changes  to  the  vehicle  suspension  system.  Its  overall  capability  remained 
the  same  as  evaluated  in  the  original  COEA.  In  essence,  the  updated  COEA  looked 
at  four  high-low  mixes  of  previously  existing  alternatives  and  their  associated  costs. 
All  four  mixed  fleet  alternatives  included  a  number  of  high  speed  amphibians 
(AAAV(F))  to  conduct  amphibious  operations.  The  analysis  also  looked  at  several 
different  development  approaches  to  obtain  mixes  of  previous  alternatives.  As  such, 
the  results  of  the  effectiveness  analysis  portion  of  the  original  COEA  did  not  change. 
The  four  "new"  alternatives  evaluated  are  described  as  follows: 

AAAV(FO):  This  mixed  fleet  alternative  included  high  speed  amphibians 
(AAAV(F))  for  amphibious  operations  and  the  current  or  baseline  system 
(AAV7A1E2)  used  for  the  remainder  of  the  Marine  Corps'  mobility 
requirements  (ex:  Maritime  Prepositioned  Ships). 

AAAV(FO+):  This  alternative  is  identical  to  the  one  previously  described  except 
that  an  upgraded  weapon  station  identical  to  the  AAAV(F)  has  been  installed  in 
the  baseline  system. 

AAAV(M):  This  alternative,  from  an  operational  effectiveness  point  of  view,  is 
a  high-low  mix  of  the  original  AAAV(F)  and  AAAV(S).  An  important  feature 
of  this  alternative  is  the  development  strategy,  in  that  the  AAAV(M)  is  a  new 
design  and  modular  in  configuration.  This  would  permit  the  conversion  of  the 
high  water  speed  amphibian  to  a  slow  water  speed  amphibian  (or  vice  versa)  at 
the  Depot  maintenance  level. 

AAAV(V):  This  alternative  results  in  the  same  capability  as  the  AAAV(M)j 
however,  it  is  achieved  through  a  series  of  five  major  block  improvements  over 
a  20+  year  period.  The  AAAV(V)  begins  with  several  block  changes  to  the 
baseline  system  and  ends  with  an  entirely  new  vehicle  capable  of  conversion 
from  high  water  speed  to  low  water  speed  configurations  (or  vice  versa). 

The  results  of  the  updated  COEA,  while  providing  several  assumption  based  options, 
did  not  come  to  definitive  conclusions.  The  Marine  Corps'  evaluation  of  the  results 
is  as  follows: 
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AAAV(FO):  While  providing,  the  most  operationally  effective  alternative  for  a 
portion  of  the  force,  this  alternative  was  rejected  for  a  number  of  significant 
reasons.  First,  the  baseline  system  in  this  mixed  fleet  option,  which  would 
represent  the  majority  of  the  mobility  assets  for  the  Marine  Corps,  is  an 
alternative  which  has  consistently  demonstrated  complete  operational 
ineffectiveness  and  is  obsolete  by  definition.  Second,  the  introduction  of  two 
vastly  different  systems  into  the  inventory  would  dramatically  compound 
training,  supply,  personnel,  maintenance,  deployment  and  operational  suitability 
problems. 

AAAV(FO+):  Again,  while  providing  the  best  alternative  for  a  portion  of  the 
force  it  was  rejected  for  the  same  reasons  as  the  AAAV(FO).  In  addition,  this 
alternative  raises  technical  concerns  in  that  the  integration  of  a  weapon  station 
with  similar  capabilities  as  other  alternatives  would  seriously  deplete  the  reserve 
buoyancy  of  the  existing  vehicle  during  water  operations  (safety).  It  would 
negatively  affect  the  vehicle's  vertical  and  longitudinal  centers  of  gravity 
(posing  stability  problems  during  water  operations),  add  significant  weight  to  a 
vehicle  whose  suspension  system  and  final  drives  are  already  overtaxed  from 
the  addition  of  existing  product  improvements,  and  cause  a  decrease  in  the 
infantry  carrying  capacity  of  the  vehicle  due  to  the  added  internal  volume  of  the 
turret  basket. 

AAAV(M):  The  principal  negative  attribute  of  this  alternative  is  the 
suboptimization  of  the  system  design  caused  by  the  requirement  to  have  it  easily 
convertible  from  one  amphibious  configuration  of  the  vehicle  to  another.  There 
is  also  an  added  technical  complexity  factor  for  a  design  which  must  be  as 
dramatically  flexible  as  this  alternative  requires.  A  mixed  fleet  of  newly 
designed  high  and  low  water  speed  amphibious  vehicles  that  share  many 
identical  systems  and  attributes  will  not  have  nearly  the  impact  on  logistics, 
maintenance,  supply,  manpower,  training,  and  operational  suitability  as 
AAAV(FO)  or  (FO+).  Some  impacts  in  these  areas  will  remain,  however,  and 
are  negative  factors. 

AAAV(V):  This  alternative  was  strongly  rejected  due  to  the  numerous  areas  of 
risk  it  presented.  The  configuration  management  risk  present  with  this 
developmental  approach  was  evaluated  as  being  exceptionally  high.  For  not 
less  than  20  years  this  system  would  be  found  in  multiple  configurations  in  the 
operating  forces  posing  training,  logistics,  maintenance  and  operational 
problems  that  would  be  devastating  to  operational  unit  readiness.  In  addition, 
the  required  levels  of  concurrence  of  developmental  activities  and  testing  also 
were  considered  high  risk.  This  approach  was  judged  to  be  highly  susceptible  to 
single  event  failure  and  overall  program  vulnerability. 
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d.  Assessment  of  Cooperative  Development  Program 

The  Marine  Corps  is  the  single  developer  and  producer  of  fully  amphibious  combat 
vehicles  in  the  free  world.  Consequently,  there  are  no  other  known  or  planned 
development  or  production  projects,  nor  are  there  any  ongoing  projects  which  could  be 
modified  to  satisfy  AAAV  program  requirements.  An  assessment  of  the  potential  for 
cooperative  development  of  the  AAAV  system  has  been  performed  resulting  in  the 
identification  of  specific  advantages  and  disadvantages  to  cooperative  development.  The 
advantages  are:  (1)  costs  would  be  reduced  as  a  result  of  cost-sharing  with  allies;  (2) 
U.S.  -  allied  Standardization  and  Interoperability  (S&I)  would  be  enhanced  through  the 
use  of  common  systems,  subsystems,  assemblies,  components  and  piece  parts;  (3) 
valuable  defense-related  technology  could  be  shared  reciprocally;  (4)  it  could  potentially 
be  the  impetus  for  broad  modernization  of  allied  amphibious  forces.  The  disadvantage  is 
that  the  uncertain  economic  and  political  position  of  nearly  all  potentially  viable  countries 
introduces  a  significant  and  unacceptable  element  of  risk  which  could  adversely  impact 
both  the  development  and  production  of  AAAVs.  However,  Foreign  Military  Sales 
(FMS)  does  offer  a  viable  alternative  to  a  Cooperative  Development  Program  that  would 
retain  the  advantages  of  improved  allied  nation  capability,  reduced  production  cost,  and 
enhanced  S&I,  all  at  minimal  risk.  Annex  G  [of  the  Integrated  Program  Summary] 
identifies  10  allied  nations  that  may  be  interested  and  could  benefit  from  AAAV  Program 
FMS.  The  AAAV  Program  Office  will  explore  this  alternative  in  place  of  a  Cooperative 
Development  Program. 

Most  Promising  Alternative  and  Rationale 

The  AAAV(F)  is  the  most  promising  and  operationally  effective  alternative  evaluated. 
Its  inherent  characteristics  provide  an  amphibious  force  with  the  ability  to  use  speed  as  a 
weapon;  through  the  rapid  concentration  of  its  available  combat  power  at  a  decisive  place 
and  time.  Other  alternatives  do  not  provide  the  velocity  of  combat  power  that  an 
amphibious  force  equipped  with  AAAV(F)  does.  All  other  alternatives  have  force 
concentration  rates  over  time  and  distance  that  are  significantly  slower,  and  thus  inhibit 
our  ability  to  seize  the  initiative  and  dictate  the  terms  of  combat  by  forcing  the  enemy  to 
react  to  us.  Without  this  initiative  an  amphibious  force's  survivability  and  security  is 
greatly  diminished,  along  with  the  loss  of  essential  elements  that  enable  maneuver  and 
surprise.  Alternatives  other  than  AAAV(F)  tend  to  move  the  amphibious  force  in  the 
direction  of  an  attrition  style  of  warfare  instead  of  in  the  direction  of  maneuver  warfare, 
which  is  the  doctrinal  foundation  for  all  U.S.  Marine  Corps  operations.  AAAV(F)  is  the 
most  promising  alternative  from  an  operational  perspective  in  that  it  is  the  most  effective 
in  achieving  military  objectives  at  the  least  cost  in  personnel  and  equipment. 
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As  outlined  above,  the  speed  at  which  a  force  is  able  to  be  concentrated  and  maneuvered 
over  sea  and  land  is  a  determining  factor  for  battlefield  success.  Of  all  the  vehicular 
alternatives  the  AAAV(F)  maneuvered  the  force  ashore  and  inland  faster  than  any  other 
when  the  number  of  LCACs  were  fixed  (as  they  presently  are).  The  surface  and  air 
options  were  comparable  to  AAAV(F)  in  this  area,  however,  they  provide  no  mobility 
once  ashore,  nor  do  they  afford  any  armor  protection  or  direct  fire  support.  Compared  to 
AAAV(F)  other  vehicular  alternatives  took  between  50%  and  300%  longer  to  build  up  a 
comparable  level  of  combat  power  ashore.  To  redress  this  significant  imbalance  in 
critical  effectiveness  other  vehicular  alternatives  would  require  the  acquisition  of 
additional  LCACs  and  very  costly  amphibious  ships  and  crews. 

The  seamlessness  in  which  the  AAAV(F)  can  maneuver  a  force  versus  the  other 
alternatives  pays  continuing  dividends  in  the  battle  ashore.  In  critical  areas  such  as  force 
movement  within  a  theater  of  operations,  the  ratio  of  enemy  forces  lost  to  friendly  forces 
lost,  and  at  the  end  of  combat  operations,  the  percent  of  friendly  forces  surviving,  the 
AAAV(F)  was  again  superior  to  all  others.  These  attributes  remained  and  in  some  cases 
increased  when  AAAV(F)  was  subjected  to  the  realities  of  combat  operations  via  the 
analytical  technique  of  sensitivity  analysis.  Alternatives  were  evaluated  in  situations 
where  enemy  arrival  time  to  landing  sites  was  faster  than  expected;  due  to  the  success  of 
enemy  deception  or  the  availability  of  less  than  perfect  intelligence.  Other  alternatives 
paid  a  consistently  heavy  price  when  this  event  occurred,  however,  AAAV(F)  actually 
improved  in  the  categories  of  loss-exchange  ratio  and  force  ratio.  The  decrease  in  total 
force  movement  using  AAAV(F)'s  in  this  situation  was  virtually  nonexistent  (less  than  .5 
percent). 

There  are  a  number  of  other  significant  reasons  why  AAAV(F)  is  the  most  promising 
alternative.  First,  it  is  an  inherently  flexible  and  multipurpose  weapon  which  can  achieve 
its  full  military  worth  without  the  need  for  auxiliary  craft  or  support  equipment.  All  other 
technically  feasible  vehicular  alternatives  require  that  their  delivery  ashore  be  supported 
by  naval  landing  craft.  These  vulnerable,  high  value  transport  systems  are  the  only 
conveyance  by  which  the  landing  forces'  heaviest  equipment  can  be  brought  ashore. 
They  are  dependent  upon  amphibious  ships  for  their  support  and  their  loss  through 
military  action  or  maintenance  casualty  will  significantly  degrade  force  effectiveness 
ashore  during  its  most  critical  stage.  Vehicular  alternatives  other  than  AAAV(F)  require 
these  landing  craft  be  placed  in  the  first  (and  all  subsequent)  assault  waves;  lending 
greater  probability  that  their  vulnerability  to  virtually  all  infantry  weapons  will  be 
exploited.  The  AAAV(F)  significantly  reduces  the  exposure  of  these  high  value  landing 
craft  to  enemy  action  as  they  do  not  require  their  use  as  a  conveyance  ashore,  and  exceed 
their  surf  transit  capability  thus  opening  up  greater  littoral  space  for  follow-on  landing 
sites.  In  addition,  a  landing  force  equipped  with  AAAV(F)  is  able  to  organically  conduct 
long  distance  shore-to-shore  and  riverine  operations  without  the  burden  of  a  significant 
land  based  landing  craft  support  structure,  or  the  dedication  of  amphibious  ship  support  at 
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sea.  The  tempo  of  operations  for  a  force  equipped  with  AAAV(F)  is  undiminished  when 
confronted  by  water  barriers  that  previously  required  deliberate  negotiation  and  the  use  of 
auxiliary  equipment  (i.e.,  landing  craft,  bridges,  barges).  Water  obstacles  abound  along 
the  littorals  of  the  world  and  can  be,  for  the  first  time,  exploited  as  high  speed  avenues  of 
approach,  rather  than  being  viewed  as  physical  barriers  which  inhibit  operational  tempo 
and  overall  speed  of  maneuver.  The  inherent  capability  of  AAAV(F)  as  both  a  high  water 
speed  amphibious  vehicle  and  armored  personnel  carrier  significantly  contributed  to  its 
assessment  as  the  most  promising  alternative. 

As  a  land  combat  vehicle  AAAV(F)  has  a  significant  advantage  over  all  other  existing, 
alternative  systems  in  terms  of  design  architecture.  Its  "open  architecture"  and  partitioned 
design  have  allowed  for  significant  growth  or  complete  change-out  of  subsystems  prone 
to  rapid  evolution  (software,  weapon  system)  or  those  that  represent  critical  risk 
(propulsion  system).  More  than  just  design  margin  or  excess  capacity,  this  open 
architecture  and  partitioned  design  philosophy  will  allow  for  greater  evolution  of  the  total 
system,  over  a  longer  vehicular  life  span,  and  at  less  cost  per  incident  of  vehicle  change. 
No  other  existing  combat  vehicle  possesses  an  inherent  system  architecture  which 
promotes  evolutionary  growth  to  the  level  of  the  AAAV(F).  As  such,  this  is  viewed  as 
additional  justification  for  its  selection  as  the  most  promising  alternative. 
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APPENDIX  H.  AAAV  SYSTEM  DESCRIPTION  - 1999  DRAFT  TEST  AND 

EVALUATION  MASTER  PLAN 


SYSTEM  DESCRIPTION 

1 .  Overview:  The  family  of  vehicles  will  consist  of  a  personnel  variant— AAAV(P),  and 
a  command  and  control  variant,  — AAAV(C).The  AAAV  Program  will  include  vehicles 
and  vehicle  sub-systems  capable  of  delivering  the  ground  combat  element  of  Marine 
assault  forces  from  amphibious  shipping  located  over-the-horizon  to  inland  objectives. 
The  AAAV  will  be  a  tracked-amphibious  vehicle  possessing  both  land  and  water 
mobility.  The  AAAV  is  designed  to  have  an  endurance  of  250  miles  after  a  high-speed 
water  march  of  25  NM.  The  estimated  weight  of  the  AAAV  is  38  tons  and  it  will  measure 
approximately  30  ft.(L)  x  12  ft.(W)  x  10  ft.(H).  The  AAAV(P)  vehicle  will  be 
operated/maintained  by  a  crew  of  3  Marines  and  have  a  troop  capacity  of  at  least  17 
Marines.  The  AAAV (C)  vehicle  will  retain  the  armored  hull,  land  and  water  propulsion 
systems  and  survivability  features  of  the  AAAV(P).  The  communication  suite  for  the 
AAAV(C)  will  be  capable  of  supporting  an  infantry  battalion  or  regimental  commander 
and  selected  staff.  The  AAAV  system  will  be  comprised  of  a  variety  of  sub-systems  that 
will  allow  the  vehicle  to  perform  its  required  operational  mission.  The  major  sub¬ 
systems  are  the  armored  hull;  land  propulsion  system;  water  propulsion  system; 
command  &  control  and  weapons  systems. 


Figure  H-l.  AAAV(P)  Representation. 
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2.  Major  Subsystems: 


a.  Armored  Hull:  The  armored  hull  of  the  vehicle  will  be  designed  to  provide 
multi-hit  protection  against  small  arms  fire  up  to  14.5mm  AP  at  300  meters.  The  hull 
will  also  protect  against  artillery  fragments  (up  to  155mm)  at  15  meters  from  any  aspect. 
The  bottom  of  the  hull  will  protect  against  anti-personnel  mines.  See  Figure  H-2. 


Figure  H-2.  “Cut  Away”  View  of  AAAV. 


b.  Land  Propulsion:  While  on  land,  the  vehicle  will  have  the  mobility 
capabilities  to  operate  with  USMC  Main  Battle  Tanks.  The  AAAV  will  have  the 
capability  to  traverse  the  same  terrain  at  the  same  speed  as  the  tank  during  cross-country 
operations  without  crew  or  troop  fatigue.  The  AAAV  will  also  cross  all  obstacles/terrain 
features  as  the  tank  (trench,  hills,  gaps,  walls,  soft  soil,  marsh,  etc.).  The  track  system  will 
operate  in  conjunction  with  the  water  propulsion  system  in  a  transition  mode  (low  water 
speed)  to  facilitate  crossing  the  surf  zone  and  fording  rivers.  See  Figure  H-3. 
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Figure  H-  3.  Land  Propulsion. 


c.  Water  Propulsion.  The  AAAV  is  powered  through  the  water  by  two  water  jets 
(see  Figure  H-4).  While  in  the  water  and  travelling  at  high  speed,  the  AAAV  will  be  a 
planing  hull  craft.  To  attain  this  high  speed,  the  AAAV  will  re-configure  its  external 
profile.  This  will  be  done  through  retraction  of  the  track  and  suspension  systems, 
deploying  cover  plates  on  the  underside  of  the  vehicle  and  extending  bow  and  transom 
flaps  to  present  a  smooth  surface.  Steering  is  provided  by  movable  steering  buckets.  In 
the  water  the  AAAV  will  be  able  to  travel  at  20  knots  (minimum)  in  Sea  State  III  (3  foot 
significant  wave  height).  At  low  speed,  in  preparation  to  traverse  the  surf  zone,  the  flaps 
are  retracted  and  the  suspension  is  lowered  and  engaged  to  operate  in  tandem  with  the 
wateijet. 


Figure  H-4.  Water  Propulsion. 


d.  Command  and  Control.  The  AAAV  is  required  to  function  within  the  Marine 
Corps  command  and  control  architecture  and  interface  and  inter-operate,  as  required,  with 
the  current  and  planned  Joint  command  and  control  architecture.  The  AAAV(P),  as  a 
integral  maneuver  support  element,  will  have  a  radio  suite  that  will  provide  line  of  sight 
(LOS)  and  beyond  line  of  sight  communications  (BLOS)  capability  to  ensure 
connectivity  with  both  higher,  adjacent  and  subordinate  maneuver/command  elements. 
The  AAAV  will  also  possess  an  intercom  system  for  communication  between  crew 
stations  and  the  embarked  infantry  commander.  The  AAAV(C)  will  have  an  integrated 
C4I  Suite  to  function  as  either  a  battalion  or  regimental  tactical  echelon  headquarters. 
The  suite  will  be  composed  of  LOS  and  BLOS  radio  equipment,  computer  hardware,  and 
tactical  software  applications.  This  C4I  Suite  will  provide  the  embarked  commander  and 
his  staff  as  the  tactical  echelon  headquarters  the  capability  to  communicate  with  and  to 
inter-operate  with  both  USMC  and  Joint  senior,  adjacent,  and  subordinate  maneuver 
units;  combat  support  units;  and,  combat  service  support  units.  Additional  detail  on  the 
interoperability  requirements  for  the  C4I  suite  for  the  AAAV  may  be  found  in  the  C4ISP. 
A  functional  interpretation  of  the  PDRR  version  of  the  AAAV(C)  C4I  suite  is  pictured  in 
Figure  H-5. 
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Figure  H-5.  AAAV(C)  PDRR  C4I  Suite. 
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e.  Firepower.  The  Mk  46  Weapon  System  comprises  the  30mm  Mk  44 
Bushmaster  II  gun  with  a  fully  stabilized  turret,  Forward  Looking  Infrared  (FLIR)  optics 
and  full  solution  fire  control  system  (see  Figure  H-6).  A  coaxial  machine  gun  will  also  be 
fitted  to  the  turret.  The  AAAV(C)’s  main  armament  will  be  a  7.62mm  or  equivalent 
machine  gun  to  provide  close  in,  self-defense  capability  against  dismounted  infantry.  The 
AAAV  weapon  suite  will  be  fully  operational  during  vehicle  operations  on  the  land  and 
in  the  water. 


Figure  H-6.  30mm  Mk  46  Weapon  System. 


3.  Software  /  Hardware  Architecture.  The  AAAV  vetronics  system  architecture  is 
comprised  of  processors,  controls,  displays  and  software  to  provide  the  interface  for 
vehicle  functions  that  are  controlled  electronically.  A  schematic  of  this  vetronics 
architecture  is  presented  in  Figure  H-7.  The  principle  components  in  the  vetronics 
architecture  include  the  following: 

-  Hull  electronics  unit  (HEU) 

-  Turret  electronics  unit  (TEU) 

-  Weapons  station  electronics  unit  (WSEU) 

-  Command  and  control  server  (CCS) 

-  Mass  memory  unit  (MMU) 

-  Slip  ring 

-  Embedded  training  server 

-  Engine  and  transmission  controllers 

-  Hull  power  distribution  unit  (HPDU) 

-  Remote  acquisition  and  control  modules  (RACM)  for  signal/power  distribution 

-  FDDI,  Utility,  and  CAN  interface  and  data  buses 

-  Displays  and  controls  for  each  crew  station 
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The  AAAV  computer  software  configuration  items  (CSCI)  and  their  principle 
functions  are  defined  as  follows  (additional  detail  may  be  found  in  the  Computer 
Resources  Life-Cycle  Management  Plan  (CRLCMP)): 

a.  Controls  and  Displays  CSCI.  The  Controls  and  Displays  CSCI  provides  the 
software  required  for  the  interface  between  the  Marine  and  the  AAAV  vehicle.  Functions 
of  the  CD  CSCI  include  mode  control,  fault  management,  menu  management, 
diagnostics,  and  Marine/Machine  interface  control. 

b.  Navigational/Situational  Awareness  CSCI.  This  CSCI  provides  the  software 
to  support  map  displays,  route  planning,  overlays,  situational  awareness,  and  reporting. 

c.  Fire  Control  CSCI.  The  FC  CSCI  software  provides  the  fire  control 
functionality  for  the  AAAV  weapons  system.  Functions  of  this  CSCI  include  ballistics 
solutions,  weapon  control,  stabilization,  sight  controller,  motor  controller,  and  fire  control 
diagnostics. 

d.  Mobility/Power  Management  and  Auxiliaries  CSCI.  The  MPA  CSCI 
software  provides  the  capability  for  control  of  the  primary  mobility,  power,  and  auxiliary 
systems  to  include  engine,  suspension,  hydraulics,  drive  train,  electric  and  auxiliary 
power,  hydrodynamic  appendages,  bilge,  NBC,  environmental  control,  and  automatic  fire 
suppression. 

e.  Embedded  Training/IETM.  This  CSCI  provides  embedded  training  for  the 
crew  positions  and  integrates  the  IETMs  with  the  diagnostics  system.  The  Embedded 
Training  Server  provides  the  interface  between  the  PCD  systems  and  various  remote  and 
appended  simulation  and  training  devices,  thus  permitting  the  AAAV  crewmember(s)  to 
train  within  the  vehicle  or  allowing  a  group  of  vehicles  to  conduct  joint  training  exercises 
from  proximal  stationary  locations.  The  AAAV  incorporates  Embedded  Training  (ET) 
capable  of  providing  simulation-based  training  in  equipment  operation,  precision  gunnery 
and  navigation,  and  full-mission  rehearsal. 

The  IETM  provides  embedded  electronic  technical  manuals  to  the  crew 
through  the  vehicle’s  controls  and  displays.  The  IETMs  are  integrated  with  the 
diagnostics  system  to  provide  the  operator  a  more  efficient  means  of  accessing  pertinent 
information. 

An  example  of  the  software  and  hardware  functional  architecture  is  shown  in 
Figure  H-7. 
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Figure  H-  7.  Hull  and  Turret  Software/Hardware  Functional  Architecture. 

f-  Software  Build  Descriptions:  The  schedule  for  software  build  deliveries  is 
shown  in  Table  II- 1  of  the  TEMP. 

(1)  Build  3.0  -  provides  updates  and/or  initial  release  of  software  to  control 
displays,  mobility  for  land  and  water,  fire  control,  hatches  and  ramp,  communications, 
navigation,  fault  management  &  self  test  and  reporting.  Builds  1 .0  and  2.0  provided 
subsystem  controls  of  engine,  power  systems  and  displays  and  were  installed  in  the 
Software  Integration  Lab(SIL)  and  PI  for  rollout. 

(2)  Build  3.1  -  updates  displays,  fault  management  &  self  test,  navigation  and 
reporting.  Provides  initial  release  of  BIT,  FIT,  Prognostics,  Diagnostics  and  IETM 
browser.  The  fidelity  of  the  ITEMs  for  this  build  is  expected  to  be  Level  IV. 

(3)  Build  4.0  -  updates  software  for  deficiencies  identified  during 
Developmental  Testing. 

(4)  Build  5.0  -  initial  release  of  C4I  integration  for  AAAV(C)  DT  Prototype. 
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(5)  Builds  6. 0/7.0  -  initial  release  of  EMD  software  for  the  AAAV(P)  and 
AAAV(C).  Level  V  IETMs  will  be  introduced  with  this  build. 

(6)  Builds  8. 0/9.0  -  updates  software  for  deficiencies  identified  during  EMD 
DT. 

4.  Interfaces.  Physical  interfaces  for  the  AAAV  require  that  it  must  be  compatible  with 
amphibious  shipping,  land,  rail  and  air  transport.  Operationally,  the  AAAV  must  be 
capable  of  communicating  with  aviation,  naval  and  ground  forces,  as  well  as 
maneuvering  with  ground  combat  elements  once  ashore.  The  AAA  Vs  will  work  in  unison 
with  other  USMC  and  United  States  Navy  (USN)  assets,  particularly  the  USMC  vertical 
assault  element  (CH-53/CH-46,  V-22),  other  surface  landing  craft  (LCAC),  and  Joint 
service  operations.  The  AAAV  must  be  supportable  and  maintainable  within  the 
capabilities  of  the  Marine  Corps  force  structure.  Both  the  AAAV(P)  and  AAAV(C) 
variants  shall  have  similar  water  and  land  mobility  characteristics.  The  AAAV  (C) 
primary  interfaces,  to  include  interoperability  (as  specified  in  the  AAAV  C4ISP),  require 
that  it  be  capable  of  communicating  and  interoperating  with  current  and  planned  Joint  and 
Marine  Corps  aviation,  naval  and  ground  force  C2  systems. 

5.  Survivability.  Figure  H-8  below  depicts  the  potential  array  of  methods  available  to 
the  AAAV  to  maximize  the  survivability  of  the  vehicle  and  its  occupants.  The  AAAV 
program  intends  to  maximize  the  use  of  most  of  the  methods  as  the  design  matures. 
Recent  survivability  studies  of  combat  vehicles  will  be  examined  for  applicability  to  the 
AAAV. 


Don’t  Be  Seen  Don't  Be  Acquired  Don't  Be  Hit  Don't  Be  Penetrated  Don't  Be  Killed 
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a.  Susceptibility  Reduction.  The  AAAV  will  use  camouflage  paint,  low-smoke 
powder  and  flash-suppression  to  reduce  the  visual  signature  of  the  vehicle.  Vehicle  shape 
will  be  modified  consistent  with  mobility  and  transport  requirements  to  minimize  the 
visual  silhouette.  Ceramic  tiles  will  be  used  to  reduce  the  vehicle’s  IR  signature  where 
appropriate.  The  vehicle  will  have  a  Rapid  Obscuration  System  (ROS)  using  expendable 
smoke  grenades  that  provide  a  quick  and  relatively  thick  layer  of  smoke  around  the 
vehicle  to  provide  visual,  IR,  and/or  MMW  protection. 

b.  Nuclear,  Biological  and  Chemical  Protection.  The  AAAV  specification 
requires  that  the  vehicle  be  Nuclear,  Biological  and  Chemical  survivable  and  will  be 
equipped  with  an  NBC  Collective  Protection  (CP)  system  and  employ  a  NBC  filtration 
system  integrated  with  an  Environmental  Control  System  .  The  NBC  CP  system  is 
designed  to  minimize  the  infiltration  of  NBC  contaminants  during  open  hatch  operations, 
resupply  and  disembarkation/embarkation  of  crew/infantry.  The  NBC  detection  and 
warning  system  (NBC  DW)  will  provide  both  interior  and  exterior  point  detection  of 
nuclear  and  chemical  contamination.  Figure  H-9  depicts  the  NBC  system  and  Figure  H- 
10  depicts  the  AAAV’s  NBC  Protection  and  Environmental  Control  System  Inlet. 


NBC  Micro  Vehicle 

Heating  /  Cooling  Commander 


Figure  H-9.  NBC  Protection  System  Integrated  into  the  AAAV. 


241 


Inlet  Shutoff  Float 


Filter  Inlet  Duct 


Water  Separator  Plate 


Water  Trap 


Water  Drain 
(To  Engine  Bilge) 


Figure  H-10.  NBC  Protection  and  Environmental  Control  System  Inlet. 


6.  Logistics.  The  maintenance  concept  for  the  AAAV  program  is  to  "fix  as  far 
forward  as  possible"  with  the  intent  of  driving  maintenance  costs  down  while  increasing 
the  availability  (or  retum-to-readiness)  of  the  vehicle.  On  a  fleet  average  basis  of  600 
engine  operating  hours  or  6  years  of  service  life,  the  AAAV  will  be  scheduled  for  depot 
level  maintenance.  Maintenance  of  the  AAAV  will  be  conducted  under  a  Two  Level 
Maintenance  Concept.  Preventive  maintenance  and  corrective  maintenance  will  be 
accomplished  at  the  lowest  appropriate  level  depending  upon  the  nature  of  the  corrective 
action  required.  Education  and  maintenance  will  be  enhanced  through  extensive  use  of 
IETMs  to  support  embedded  training  and  diagnostics.  Replacement  of  failed  or  worn 
components  will  be  accomplished  on  a  piece-part  basis  at  the  appropriate  level  depending 
upon  the  requirements  for  special  tools,  special  procedures,  or  special  skills  necessary  to 
accomplish  the  required  corrective  action. 
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APPENDIX  I.  AAAV  CRITICAL  TECHNICAL  PARAMETERS  - 1999  DRAFT 

TEST  AND  EVALUATION  MASTER  PLAN 
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NOTES  ARE  PROVIDED  AT  END  OF  APPENDIX . 


Critical  Technical  Parameter 


ORD  para.  4.a.(2)(c)  (^fl  ^WH)  |  |  1  ,  - . — 

will  be  evaluated  by  analysis  and  extrapolation  of  data  galhercd  from  performance  of  .he  AAA  v  m  :>ea  wk  111  ami  lower. 


Technical 

Test  Threshold  for  Decision  Demonstrated 

Critical  Technical  Parameter  Events  Each  Event  Location  Schedule  Supported  Value 
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NOTES  ARE  PROVIDED  AT  END  OP  APPENDIX 


Provide  secure  inter-vehicle  and  radio  frequency  DT-I  AVTB  FYOO  Milestone  II 

transmit/receive  communications  capability 

ORD  para.  4x\(2)  DT-II  NOTE  10  TBD  FY03-05  Milestone  III 
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Technical 

Total  Threshold  for  Decision  Demonstrated 

Critical  Technical  Parameter  Events  Each  Event  Location  Schedule  Supported  Value 
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Support,  Mission  Planning  and  Rehearsal,  and  Logistics  applications  fielded 

to  infantry  battalions 
and  regiments. 

_ ORP  para.  4.a.(9)(c)  _  NOTE  15 _ 


NOTES  ON  CRITICAL  TECHNICAL  PARAMETERS 

NOTE  1:  NUCLEAR,  BIOLOGICAL  AND  CHEMICAL  SURVIVABILITY 

As  stated  in  the  ORD: 

NBC  Survivability.  The  AAAV,  crew,  and  embarked  personnel  must 
be  able  to  survive  and  operate  in  an  NBC  Mission  Oriented  Protective 
Posture  (MOPP)  4  environment.  Protection  for  the  crew  and  embarked 
Marines  is  mandatory.  The  vehicle  also  must  be  able  to  withstand  the 
corrosive  effects  of  all  known  chemical  warfare  hazards  and  standard 
decontaminants.  Radiological  protection  will  be  provided  by  the 
ballistic  hull  and  the  vehicle  will  be  unaffected  by  initial  neutron  and 
gamma  radiation  levels  below  those  that  will  cause  human  casualties. 
Protection  against  electromagnetic  pulse  produced  by  nuclear 
detonations  and  the  accumulation  of  residual  radiation  hazards  is  a 
threshold  requirement.  The  AAAV  shall  include  an  automatic  chemical 
agent  and  a  radiation  alarm  and  measurement  system.  These  systems 
will  be  able  to  measure  these  effects,  both  inside  and  outside  the 
vehicles,  providing  chemical  agent  identification  that  alerts  and 
measures  the  chemical  and  radiation  effects. 

The  type  of  NBC  Collective  Protection  (CP)  system  (includes  environmental  control 
unit)  and  NBC  detection/alarm  equipment  to  be  incorporated  into  the  AAAV  shall  be 
subject  to  US  Army  development  efforts.  Any  NBC  systems/components  used  on  the 
AAAV  shall  be  non-developmental  to  the  DRPM  AAA. 

The  AAA V's  NBC  survivability  criteria  has  been  developed  and  is  specified  in 
references  42  and  43,  of  Appendix  A.  Requirements  and  specifications  are  not  listed 
herein  due  to  SECRET  classification.  These  requirements  shall  be  detailed  in  classified 
Detailed  Test  Plans  (DTP).  During  the  PDRR  phase,  the  vehicle  mounted  NBC  CP 
system  and  NBC  detection/alarm  system  shall  be  tested  using  threat  chemical  agent 
simulants.  Stand-alone  NBC  CP  systems  and  their  components,  and  the  stand-alone  NBC 
detection/alarm  systems  and  their  components  shall  be  tested  using  live  chemical  agents. 
During  the  EMD  phase,  the  final  versions  of  the  vehicle  mounted  NBC  CP  system  and 
the  NBC  detection/alarm  system  shall  be  tested  using  threat  chemical  agent  simulants. 
Final  versions  of  stand  alone  NBC  CP  systems  and  their  components,  and  NBC 
detection/alarm  systems  and  their  components  shall  be  tested  using  live  chemical  agents. 
The  Government  shall  approve  all  live  chemical  agents  used  for  testing,  and  all  testing 
shall  be  accomplished  within  DoD  and  environmental  guidelines  and  practices. 
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The  AAAV  will  be  tested  to  ensure  it  meets  the  Nuclear,  Biological,  and  Chemical 
Contamination  Survivability  (NBCCS)  criteria  identified  in  Department  of  the  Army, 
Office  of  the  Deputy  Chief  of  Staff  for  Operations  and  Plans  memorandum,  dated  24  Oct 
91,  DEPARTMENT  OF  THE  ARMY  APPROVED  QUANTITATIVE  NBC 
CONTAMINATION  SURVIVABILITY  CRITERIA.  During  the  PDRR  phase  the 
AAAV  will  be  tested  for  compatibility  of  operation  and  decontamination,  using 
decontamination  simulants,  by  personnel  in  full  NBC  protective  ensemble  (MOPP  4). 
AAAV  armor/hull  samples  will  be  tested  for  NBCCS  hardness  and  decontaminability 
using  live  chemical  agents  and  actual  decontaminates.  During  the  EMD  phase,  personnel 
in  full  NBC  protective  ensemble  (MOPP  4)  will  test  the  AAAV  for  compatibility  of 
operation.  If  the  AAAV's  armor/hull  material  changes  from  that  tested  during  the  PDRR 
phase  then  samples  of  the  new  material  will  be  tested  for  NBCCS  hardness  and 
decontaminability  using  live  chemical  agents  and  actual  decontaminates. 

The  AAAV's  Initial  Nuclear  Weapons  Effects  (INWE)  survivability  criteria  has  been 
developed  and  is  specified  in  a  classified  document  listed  in  Appendix  A.  Requirements 
and  specifications  are  not  listed  herein  due  to  SECRET  classification.  These 
requirements  shall  be  detailed  in  classified  DTPs.  During  the  PDRR  phase,  AAAV 
components  will  be  screened,  analyzed  and  in  some  cases  tested  to  determine  their 
Nuclear  Hardness.  During  the  EMD  phase  the  AAAV,  as  a  complete  system,  will  be 
analyzed  and  tested  to  verify  its  hardness  against  INWE.  Testing  and  evaluation  shall  be 
conducted  to  verify  that  nuclear  hardness  is  maintained  during  both  production  and 
deployment,  and  that  vulnerabilities  are  not  introduced  through  changes  in  components  or 
in  manufacturing  methods.  Parameters  that  shall  be  evaluated  to  determine  and/or  verify 
the  AAAV's  nuclear  hardness  are:  Structural  Effects  (overpressure),  Thermal  Effects, 
Ionizing  Radiation  Effects,  (gamma  rays  and  neutron  effects),  and  Electromagnetic  Pulse 
effects,  including  Transient  Radiation  Effects  (TRE)  on  Electronics. 


NOTE  2:  OBSCURATION  PROJECTION 

Trade-off  analyses  have  been  performed  for  smoke  system  selection  and  will  be 
updated  in  the  PDRR  phase.  Current  requirements  allow  for  use  of  several  technologies 
currently  employed  on  US  combat  vehicles  (such  as  fog  oil-injected  exhaust  systems  and 
reloadable  grenade  launchers  capable  of  projecting  millimeter  wave  and  multi-spectral 
smoke). 
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NOTE  3:  SUSTAINED  CROSS-COUNTRY  SPEED 


The  requirement  for  the  AAAV  system  during  land  operations  is  to  be  capable  of 
operating  cross-country  and  in  sustained  operations  with  the  Main  Battle  Tank  (MBT)  of 
the  time  frame.  Two  test  parameters  will  be  measured:  ability  to  sustain  the  required 
speed  for  the  given  terrain,  and  the  capability  for  the  crew  and  embarked  troops  to  be 
combat  effective  after  cross-country  operations.  The  evaluation  of  personnel  not 
succumbing  to  cross  country  vibration  and  shock  loading  will  be  in  accordance  with 
Military  Standards  and  established  US  Army  mobility  evaluation  procedures  (6- watt  ride 
limiting  speed  and  Fatigue  Decreased  Proficiency  Level  in  accordance  with  Mil-Std- 
1472D).  These  procedures  evaluate  operating  speeds  from  zero  speed  to  maximum 
attainable  speed  over  specified  bumps  and  random  terrain  courses  (measured  RMS 
profiles).  Accelerations  and  absorbed  power  levels  at  selected  vehicle  crew  or  troop 
stations  are  the  measured  data.  During  ATR  testing,  DT-I  and  DT-II,  test  measurements 
will  focus  first  on  vehicle  speed  capability,  and  secondly  on  ride  quality/power  absorption 
evaluation  across  the  vehicle  speed  range. 


NOTE  4:  HIGH  WATER  SPEED  IN  SEA  STATE  III 

Over-water  vehicle  speed  and  ride  quality  (fatigue  decreased  proficiency  limits  and 
shock  loads)  will  be  measured  in  the  highest  attainable  sea  conditions,  up  to  the  Sea  State 
III  limit. 

Sea  conditions  of  3-foot  significant  wave  height  in  a  Pierson-Moskowitz  random  sea 
spectrum  are  critical  criteria.  Each  contractor's  HTR  has  already  surpassed  the  speed 
requirement  (scaled  for  the  vehicle  size  and  power  available).  Test  scheduling, 
prioritization  and  asset  allocation  will  be  directed  toward  attaining  sea  conditions  during 
DT-I  and  DT-II.  Test  sites  exist  on  the  East  Coast  and  West  Coast  to  attain  sea 
conditions. 


NOTE  5:  NEGOTIATE  PLUNGING  SURF 

Every  effort  will  be  made  to  test  at  the  full  Sea  State  requirement,  but  the  ability  to 
test  in  the  designated  sea  state  condition  will  be  dependent  on  environmental  conditions 
at  the  time  of  testing.  Test  scheduling,  prioritization  and  asset  allocation  will  be  directed 
toward  attaining  sea  conditions  during  DT-I  and  DT-II.  Test  sites  exist  on  the  East  Coast 
and  West  Coast  to  attain  surf  conditions. 
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NOTE  6:  OPERATION  IN  HEAVY  SEAS 


The  AAAV,  in  a  combat  loaded  condition  shall  “survive”  in  Sea  State  V  (10  ft. 
significant  wave  height).  To  survive,  the  AAAV,  with  power,  must  be  able  to  maintain 
heading.  Without  power,  it  must  not  capsize  for  at  least  8  hours. 


NOTE  7:  RELIABILITY,  AVAILABILITY,  MAINTAINABILITY,  AND 
DURABILITY  (RAM-D) 

Testing  of  RAM-D  parameters  will  allow  for  maturation  of  components  for  the  EMD 
phase.  Data  collected  during  DT-I/OT-I  will  be  utilized  to  verify  and  adjust  RAM-D 
engineering  analyses  conducted  during  PDRR.  Dedicated  reliability  growth  testing  will 
be  conducted  in  EMD  based  on  test  results  from  DT-I  and  EOA.  Since  LRIP  vehicles 
will  be  delivered  over  a  24-month  period,  RAM-D  testing  conducted  early  in  EMD  will 
impact  LRIP  vehicle  design.  In  addition,  later  testing  of  EMD  production  representative 
vehicles  during  DT-II,  IOT&E,  and  LRIP  vehicles  during  FOT&E  will  be  reflected  in 
full-rate  production.  A  Test,  Fix,  and  Re-test  strategy  will  be  utilized  to  fully  mature 
RAM-D  parameters.  Table  1-1  depicts  the  verification  method  for  the  key  RAM-D 
parameters.  Although  Operational  Availability  (Ao)  is  specified  in  the  ORD,  only 
Inherent  Availability  (Ai)  will  be  testable  (Ao  may  be  estimated).  This  is  due  to  logistics 
administration  (MPF-Future  parts  block,  Sea-based  Logistics  stockpile  locations,  Just-In- 
Time  transit  time,  etc)  that  cannot  be  tested,  but  is  only  experienced  once  the  fielded 
system  is  at  full  operational  level  and  tempo.  Since  durability  requirements  are  very 
large,  conducting  a  statistically  confident  test  is  not  economically  feasible.  Thus 
analyses,  supported  by  test  data,  will  be  utilized  to  verify  these  requirements. 


Table  1-1.  RAM-D  Requirements 


RAM-D  Parameter 

PDRR 

EMD 

LRIP 

MCMTo  <  1 .5  hours 

Analysis 

Test 

MCMT;  <12  hours 

Analysis 

Test 

70  hours  <  MTBOMF  95  hours 

Analysis 

Test 

MR  <  0.6 

Analysis 

Analysis 

Test 

0.80  <  Ao  <  0.85 

Analysis 

Analysis 

Analysis 

MTBMA  >  20  hours 

Analysis 

Analysis 

Test  i 

Durability  >  600  hours  or  6  years,  at  0.9 
probability 

Analysis 

Analysis 

Analysis  1 
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NOTE  8:  OPERATE  IN  VARIED  TERRAINS  AND  EXTREME  TEMPERATURE 
ENVIRONMENTS 

Due  to  limited  test  assets  during  the  PDRR  phase,  low  temperature/arctic  testing  and 
jungle/  tropic  environment  testing  will  be  accomplished  in  test  chambers  or  on  selected 
components  during  DT-I.  Scheduling  of  DT-I  test  events  precludes  the  utilization  of  the 
environmental  chambers  prior  to  the  MS  II  review.  Chamber  testing  is  planned  in  the 
second  or  third  quarter  of  FY01.  Testing  in  a  more  operational  environment  will  be 
conducted  during  DT-II.  Thresholds  do  not  exist  for  this  parameter,  except  that  varied 
terrain  and  climates  shall  not  adversely  impact  operation  and  maintenance  of  the  vehicle 
system. 


NOTE  9:  POSITION  LOCATION  AND  HEADING  REFERENCE 

The  AAAV  will  utilize  a  Global  Positioning  System  (GPS)  and  the  current  USMC 
Position  Location  and  Reporting  System  (PLRS)  as  a  part  of  a  total  navigation  system,  in 
addition  to  other  proposed  navigation  means  as  proposed  by  the  contractor  and 
incorporated  in  the  vehicle  design.  The  following  parameters  in  Table  1-2  are  tentative 
conditions  for  testing  that  shall  be  used  for  both  land  and  water  position  location  and 
heading  test  measurement: 


Table  1-2.  Position  Location/Heading  Test  Parameters 


Acquisition 

Phase 

Land  Position 
Accuracy 

Land  Heading 
Accuracy 

Water  Position 
Accuracy 

Water  Heading 
Accuracy 

PDRR 

+  5  degrees 

+  5  degrees 

EMD 

* 

+  4  degrees 

+  4  degrees 

LRIP 

* 

+  3  degrees 

* 

+  3  degrees 

♦Land  and  Water  Position  Accuracy  will  be  established  upon  information  received  from  the  results  of  PDRR 


Testing  to  be  done  by  the  Government  with  other  systems  (PSD  vehicle  mounted  with 
GPS,  PLRS,  and  magnetic  heading  devices)  will  investigate  needs  and  requirements  for 
navigation,  position  location  and  driver  updates. 
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NOTE  10:  COMMUNICATIONS 


Determination  of  parameters  and  selection  of  communication  sub-systems  shall  be 
dependent  on  availability  of  military  systems  of  the  time  frame  (NDI  to  the  Marine 
Corps).  Testing  to  determine  the  effect  of  the  composite  hull  on  the  ground  plane  will  be 
accomplished  (as  required). 


NOTE  11:  EMBEDDED  COMPUTER  RESOURCES 

COMPUTER  SOFTWARE 

Determination  of  parameters  and  testing  of  software  shall  be  dependent  on  the 
development,  selection,  and  incorporation  of  specific  components  in  the  vehicle.  During 
PDRR,  MIL-STD-498  shall  be  used  for  the  software  development  with  an  evolutionary 
approach  consisting  for  four  builds.  Formal  testing  of  each  build  is  planned  at  the 
conclusion  of  each  subsequent  build  (e.g.,  formal  testing  of  build  one  is  planned  at  the 
conclusion  of  build  two).  Unless  otherwise  approved  by  the  Government,  all  software 
will  be  developed  in  Ada  in  accordance  with  guidelines  established  in  IEEE  Std  990- 
1987,  Recommended  Practice  for  Ada  as  a  Program  Design  Language,  and  ANSI/MIL- 
Std  181 5 A,  Ada  Programming  Language.  The  contractor  shall  establish,  control,  and 
maintain  a  software  engineering  environment  to  perform  the  software  engineering  effort. 
As  a  minimum,  this  environment  shall  consist  of  modem  integrated  software  engineering 
tools  to  assist  with  such  things  as  requirements  tracking  and  traceability,  software 
metrics,  and  configuration  management.  A  shadow  database,  consisting  of  data  contained 
in  these  tools,  shall  be  available  for  on-line  access  via  a  direct  connection  with  DRPM 
AAA  throughout  the  software  development.  Also  contained  in  the  engineering 
environment  will  be  a  mock-up  configuration  of  the  vehicle  electronics  to  serve  as  a 
software  integration  and  testbed.  Actual  vehicle  components  or  subsystems  shall  be  used 
to  the  greatest  extent  possible.  Finally,  DRPM  AAA  shall  deploy  an  Independent 
Verification,  Validation,  and  Accreditation  (IVV&A)  team  throughout  the  development 
process.  The  Marine  Corps  Tactical  System  Support  Activity  (MCTSSA)  is  the  USMC's 
software  support  activity  (SSA)  and  will  assume  configuration  control  of  the  AAAV 
software  once  the  AAAV  is  deployed. 


COMPUTER  RESOURCE  RESERVE 

Computer  resource  reserve  testing  methods  and  tools  will  depend  on  the  selected 
hardware  platforms  and  data  busses.  It  is  expected  that  a  combination  of  software  and 
hardware  analyzers  shall  be  utilized  to  test  the  resource  reserve  capacities  at  peak  (full 
operational)  loading.  Table  1-3  depicts  the  parameters  required  for  testing  during  the 
various  acquisition  phases. 
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Table  1-3.  Computer  Resource  Reserve 


Parameter 

PDRR 

EMD/LRIP 

Computer  Processor  Throughput 

200%' 

200% 

Master  Data  Bus 

500% 

500% 

Digital  Storage 

100%' 

500% 

Memory 

200%' 

200% 

I/O  Channels 

60%'  ■ 

60% 

1  Reserve  percentages  are  required  at  the  CSCI  level. 


NOTE  12:  EFFECTIVE  FIREPOWER  EMPLOYMENT  WHILE  MOVING  AND 
STATIONARY 

Test  measurements  will  be  structured  in  two  separate  phases,  and  the  results 
combined  to  evaluate  the  ability  of  the  AAAV  test  vehicles  and  their  weapon  systems  to 
meet  the  requirement.  Quantification  of  these  parameters  is  reflected  in  the  STA  and  the 
ORD. 

Prior  to  any  lethality  being  effected  on  the  threat,  the  AAAV  system  must  possess  a 
level  of  accuracy  and  capability  to  deliver  the  threat  defeating  mechanism  onto  the  target 
at  the  required  range.  The  ORD  states  a  cumulative  probability  of  hit  (PHcum=0.90  at  1500 
meters)  accuracy  level  that  will  be  tested.  This  accuracy  will  be  determined  for  the 
weapon  system  that  is  mounted  on  the  host  (test)  vehicle.  Engagement  of  stationary  and 
moving  targets  (threat  light  armored  vehicles)  from  a  stationary  and  moving  host  vehicle 
will  be  tested. 

The  AAAV  weapon  system  must  also  be  capable  of  penetrating  the  equivalent 
thickness  of  armor  as  stated  in  the  STA  {classified)  for  the  worst  case  threat  vehicle. 
Ability  to  meet  both  the  accuracy  measure  and  lethality  measure  (although  conducted 
separately)  will  constitute  acceptance  that  the  as-mounted  weapon  system  can  defeat  the 
threat  light  armored  vehicles  of  the  time  frame  at  the  prescribed  distance.  Failure  to  meet 
either  measure  constitutes  unacceptable  performance.  Detailed  test  plans  will  address 
system  level  tests  to  validate  any  GFE/GFP  subsystem  performance  when  integrated  into 
the  AAAV  (such  as  weapon  systems). 
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NOTE  13:  COMMANDER/GUNNER  VISION 

The  crew  station  for  the  Vehicle  Commander/Gunner  must  possess  the  following 
attributes: 


Capable  of  360°  Field  of  Regard  in  horizontal  plane 

Day/night  and  all-weather  vision  (to  the  effective  range  of  the  weapon) 

FOV  in  the  vertical  plane  equal  to  or  better  than  weapon  system  elevation 
and  depression  limits  (60  degrees  elevation,  10  degrees  depression  over 
front  of  vehicle,  zero  degrees  depression  over  rear  of  vehicle  required) 

It  is  anticipated  no  growth  is  needed  for  these  criteria  -  all  delivered  vehicle  systems 
will  provide  inherent  compliance. 


NOTE  14:  SELF-DEFENSE  CAPABILITY  -  AAAV(C) 

The  requirement  for  this  capability  is  defined  as  an  anti-personnel,  self-defense 
weapons  system.  It  is  expected  that  a  weapon  equivalent  to  a  7.62mm  machine  gun  with 
a  360°  field  of  fire  will  be  employed  on  the  AAAV(C). 


NOTE  15:  AAAV(C)  C4I  SUITE  CAPABILITY 

The  AAAV(C)  C4I  Suite  will  be  an  integrated  system  composed  of  specified  tactical 
communications  equipment,  MAGTF  C2  tactical  software  applications,  and  selected  C2 
systems  equipment  that  will  accommodate  the  identified  tactical  software  applications. 
Testing  will  be  conducted  on  the  capability  of  this  integrated  system  to  provide  the  RF 
and  LAN  connectivity  required,,  sufficient  command/decision  support  processing  to  the 
embarked  commander  and  his  staff,  and  effective  and  supportable  C2  equipment  for  the 
identified  tactical  applications. 

The  C4I  Suite  will  be  tested  using  a  representative  activity  loading.  The  loading  will 
be  provided  through  operational  message  text  via  a  TOEL.  This  will  ensure  that  both 
internal  and  external  communications  media  will  be  exercised  and  that  sufficient 
throughput/bandwidth  are  available  to  support  a  Regimental  or  Battalion  Commander  and 
Staff.  Further,  this  loading  will  ensure  interoperability  with  external  systems/devices, 
provide  necessary  input  to  the  software  applications  resident  in  the  C4I  Suite,  stimulate 
interoperability  and  appropriate  command/decision  aid  processing,  and  illicit  appropriate 
commander  and  staff  interaction  within  the  AAAV(C)  Command  Post. 
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APPENDIX  J.  LIST  OF  ACRONYMS 


A 

A0 

AAA 

AAV 

AAV7A1 

AAV(C)7A1 

AAV(P)7A1 

AAV(R)7A1 

AAAV 

AAAV(P) 

AAAV  (C) 

AAS 

ACAT 

ADM 

AFV 

AoA 

AP 

APB 

APC(X) 

APG 

ASN  (RD&A) 

ATC 

ATGM 

ATR 

ATR(C) 

AVTB/D 

B 

BDAR 

BHT/BH&T 

BRAC 

BTR/B  RDM/BMP 
C 

C4I 

CAE 

CAX 

CDRL 


Availability  (Operational) 

Advanced  Amphibious  Assault 

Assault  Amphibian  Vehicle 

Assault  Amphibious  Vehicle,  Model  7A1 

Assault  Amphibious  Vehicle,  Model  7A1-  Command  and  Control 

Variant 

Assault  Amphibious  Vehicle,  Model  7A1-  Personnel  Variant 

Assault  Amphibious  Vehicle,  Model  7A1-  Recovery  Variant 

Advanced  Amphibious  Assault  Vehicle 

AAAV  -  Personnel  Variant 

AAAV  -  Command  and  Control  Variant 

Assault  Amphibian  School 

Acquisition  Category 

Acquisition  Decision  Memorandum 

Armor  Family  of  Vehicles 

Analysis  of  Alternatives 

Armor  Piercing 

Acquisition  Program  Baseline 

Armored  Personnel  Carrier  (Experimental) 

Aberdeen  Proving  Ground 

Assistant  Secretary  of  the  Navy  for  Research,  Development  & 

Acquisition 

Aberdeen  Test  Center 

Anti-Tank  Guided  Missile 

Automotive  Test  Rig 

Automotive  Test  Rig  -  Command  and  Control  Variant 
Amphibian  Vehicle  Test  Branch/Directorate 


Battle  Damage  Assessment  and  Repair 
Ballistic  Hull  and  Turret 
Base  Realignment  and  Closure 
Threat  Combat  Vehicles 


Command,  Control,  Communications  and  Computers 
Component  Acquisition  Executive 
Combined  Arms  Exercise 
Contract  Data  Requirement  List 
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CD/V 

CE 

CE/D 

CJCS 

COEA 

COI 

CP 

CPAF 

CPX 

CTP 


Concept  Demonstration  and  Validation  Phase 

Concept  Exploration  Phase 

Concept  Exploration/Definition  Phase 

Chairman,  Joint  Chiefs  of  Staff 

Cost  and  Operational  Effectiveness  Analysis 

Critical  Operational  Issue 

Command  Post 

Cost  Plus  Award  Fee  Contract 

Command  Post  Exercise 

Critical  Technical  Parameters 


D 

DA 

DAB 

DAE 

DemVal 

DIA 

DoD 

DOT&E 

DOTE  (LFT) 

DRB 

DRPMAAA 

DS&TS 

DS&TS(DT) 

DSMC 

DT 

DT-I 

DT-II 

DT&E 

DTIC 

DTP 

DTSE&E 

D/V 


Department  of  the  Army 

Defense  Acquisition  Board 

Defense  Acquisition  Executive 

Demonstration  and  Validation  Phase 

Defense  Intelligence  Agency 

Department  of  Defense 

Director  Operational  Test  and  Evaluation 

Director  Operational  Test  and  Evaluation  (Live  Fire  Test) 

Defense  Resource  Board 

Direct  Reporting  Program  Manager,  Advanced  Amphibious 
Assault 

Director  Strategic  and  Tactical  Systems 

Director  Strategic  and  Tactical  Systems  (Developmental  Testing) 

Defense  Systems  Management  College 

Developmental  Testing 

Developmental  Testing  during  PDRR  Phase 

Developmental  Testing  during  EMD  Phase 

Developmental  Test  and  Evaluation 

Defense  Technical  Information  Center 

Detailed  Test  Plan 

Director  Test  Systems  Engineering  and  Evaluation 
Demonstration  and  Validation  Phase 


E 

EAAK 

EMC 

EMI 

EMD 

EMT 

EOA 


Enhanced  Armor  Applique  Kit 

Electro-Magnetic  Compatibility 

Electro-Magnetic  Interference 

Engineering  and  Manufacturing  Development  Phase 

Executive  Management  Team 

Early  Operational  Assessment 
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F 


FAR 

Federal  Acquisition  Regulation 

FASA 

Federal  Acquisition  Streamlining  Act 

FAT 

First  Article  Test 

FCT 

Foreign  Comparative  Testing 

FD/SC 

Failure  Definition/Scoring  Criteria 

FIFV 

Future  Infantry  Fighting  Vehicle 

FMC 

Food  and  Machinery  Corporation 

FMF 

Fleet  Marine  Force 

FOC 

Full  Operational  Capability 

FOF  RTCA 

Force-on-Force  Real-Time  Casualty  Assessment 

FOT&E 

Fdllow-On  Operational  Test  and  Evaluation 

FRP 

Full  Rate  Production 

FSD 

Full  Scale  Development  Phase 

FUSL 

Full  Up  System  Live  Fire  Test 

FY 

Fiscal  Year 

G 

GAO 

General  Accounting  Office 

GD 

General  Dynamics 

GDAMS 

General  Dynamics  Amphibious  Systems 

GDLS 

General  Dynamics  Land  Systems 

H 

HE 

High  Explosive 

HTR 

Hydrodynamic  Test  Rig 

HWSTD 

High  Water  Speed  Technology  Demonstrator 

I 

IER 

Independent  Evaluation  Report 

I-IPT 

Integrating  Integrated  Product  Team 

ILS 

Integrated  Logistic  Support 

IOC 

Initial  Operational  Capability 

IOT&E 

Initial  Operational  Test  and  Evaluation 

IPPD 

Integrate  Product  Process  Development 

IPT 

Integrated  Product  Team 

IROAN 

Inspect  and  Repair  Only  As  Necessary 

J 

JITC 

Joint  Interoperability  Test  Command 
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JROC 

JMEM 

K 

kph 

KPP 

L 

LAV-25 

LCAC 

LER 

LF 

LFM  0-1 

LFT 

LFT&E 

LFT&E  WG 

LRIP 

LVA 

LVT 

LVT(X) 

M 

$M 

m 

mm 

MAA 

MAGTF 

MAIS 

MARCORSYSCOM 

MBT 

MCAGCC 

MCCDC 

MCOTEA 

MCRDAC 

MDA 

MDAP 

MNS 

MOE 

MOP 

MPS 

mph 

MRV 


Joint  Requirements  Oversight  Counsel 
Joint  Munitions  Effectiveness  Manual 


Kilometers  per  hour 
Key  Performance  Parameter 


Light  Armored  Vehicle  (USMC  operated) 
Landing  Craft  Air  Cushion 
Loss  Exchange  Ratio 
Live  Fire 

Landing  Force  Manual  0-1 
Live  Fire  Test 

Live  Fire  Test  and  Evaluation 

Live  Fire  Test  and  Evaluation  Working  Group 

Low-Rate  Initial  Production 

Landing  Vehicle,  Assault 

Landing  Vehicle,  Tracked 

Landing  Vehicle,  Tracked  (Experimental) 


Million,  Dollars 

Meters 

Millimeters 

Mission  Area  Analysis 

Marine  Air  Ground  Task  Force 

Major  Automated  Information  System 

Marine  Corps  Systems  Command 

Main  Battle  Tank 

Marine  Corps  Air  Ground  Combat  Center 

Marine  Corps  Combat  Development  Command 

Marine  Corps  Operational  Test  and  Evaluation  Activity 

Marine  Corps  Research  and  Development  Command 

Milestone  Decision  Authority 

Major  Defense  Acquisition  Program 

Mission  Needs  Statement 

Measures  of  Effectiveness 

Measure  of  Performance 

Maritime  Proposition  Shipping 

Miles  per  hour 

Mission  Role  Variant 
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MSC 

Military  Sealift  Command 

M&S 

Modeling  and  Simulation 

N 

N/A 

Not  Applicable 

NAE 

Naval  Acquisition  Executive 

NAS 

Naval  Air  Station 

NAVSEA 

Naval  Sea  Systems  Command 

NBC 

Nuclear,  Biological,  and  Chemical 

NDI 

Non-Developmental  Item 

NSC 

National  Security  Committee 

NTC 

National  Training  Center 

O 

OA 

Operational  Assessment 

OAT 

Office  of  Advanced  Technology 

O-IPT 

Overarching-Integrated  Product  Team 

OMB 

Office  of  Management  and  Budget 

OMFTS 

Operational  Maneuver  From  The  Sea 

OMFTS/STOM 

Operational  Maneuver  From  The  Sea  /  Ship-To-Objective- 
Maneuver 

OMP 

Operational  Mission  Profile 

OMS/MP 

Operational  Mode  Summary/Mission  Profile 

ONR 

Office  of  Naval  Research 

ORD 

Operational  Requirements  Document 

OSD 

Office  of  the  Secretary  of  Defense 

OT 

Operational  Testing 

OTA 

Operational  Test  Activity 

OT&E 

Operational  Testing  and  Evaluation 

OTH 

Over  the  Horizon 

P 

PI 

AAAV  PDRR  Prototype  #1 

P2 

AAAV  PDRR  Prototype  #2 

P3 

AAAV  PDRR  Prototype  #3 

PAT&E 

Production  Acceptance  Test  and  Evaluation 

PDM 

Program  Decision  Memorandum 

PDRR 

Program  Definition  and  Risk  Reduction  Phase 

PEO 

Program  Executive  Officer 

PF/DOS 

Production,  Fielding/Deployment,  Operations  and  Support  Phase 

PIP 

Product  Improvement  Program 

PM 

Program  Manager 
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PMAAV 

PMO 

PMT 

Pnp 

POM 

PSD 

PVT 


Program  Manager,  Assault  Amphibian  Vehicle 

Program  Management  Office 

Program  Management  Team 

Probability  of  No  Penetration 

Program  Objective  Memorandum 

Propulsion  System  Demonstrator 

Product  Verification  Testing 


R 

R&D 

R&M 

RAM 

RAM-D 

RAM/RS 

RDT&E 

RFP 

ROC 

RPG 


Research  and  Development 
Reliability  and  Maintainability 
Reliability,  Availability,  Maintainability 
Reliability,  Availability,  Maintainability  and  Durability 
Reliability,  Availability,  Maintainability/Retum  to  Standard 
Research,  Development,  Test  &  Evaluation  Funding 
Request  For  Proposal 
Required  Operational  Capability 
Rocket  Propelled  Grenade 


S 

SCP 

SCRE 

SECDEF 

SEDS 

SDR 

SLEP 

SOA 

SOW 

SOP 

SRR 

ss 

s/ss 

STEP 

STU 


System  Concept  Paper 
Stratified  Charged  Rotary  Engine 
Secretary  of  Defense 
System  Engineering  Design  Schedule 
System  Design  Review 
Service  Life  Extension  Program 
Sustained  Operations  Ashore 
Statement  Of  Work 
Standard  Operating  Procedure 
System  Requirements  Review 
Sea  State 

System/Subsystem 

Simulation,  Test  and  Evaluation  Process 
Special  Trials  Unit 


T 

T&E 

Test 

TEMP 

Test 

TIWG 

Test 

TPD 

Test 

and  Evaluation 
and  Evaluation  Master  Plan 
Integration  Working  Group 
Planning  Document 
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u 


UDLP 

United  Defense  Limited  Partnership 

UNDEX 

Underwater  Explosion 

USA 

United  States  Army 

use 

United  States  Code 

USD  (A&T) 

Under  Secretary  of  Defense  for  Acquisition  and  Technology 

USMC 

United  States  Marine  Corps 

V 

VCJCS 

Vice  Chairman  Joint  Chiefs  of  Staff 

VDD 

Virtual  Design  Database 

VV&A 

Verification/Validation  and  Accreditation 

w 

WBS 

Work  Breakdown  Structure 

WIPT 

Working  Level  Integrated  Product  Team 

WWII 

World  War  Two 
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56.  Direct  Reporting  Program  Manager  Advanced  Amphibious  Assault,  Acquisition 
Plan  No.:  96-1  Rev  02,  21  October  1997. 

57.  Direct  Reporting  Program  Manager  Advanced  Amphibious  Assault,  Draft  Test  and 
Evaluation  Master  Plan,  30  August  1999. 

58.  Direct  Reporting  Program  Manager  Advanced  Amphibious  Assault,  Draft  Test  and 
Evaluation  Master  Plan,  4  November  1988. 

59.  Direct  Reporting  Program  Manager  Advanced  Amphibious  Assault,  Test  and 
Evaluation  Master  Plan,  8  July  1994. 
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60.  Direct  Reporting  Program  Manager  Advanced  Amphibious  Assault,  Integrated 
Program  Summary,  January  1995. 

61.  Direct  Reporting  Program  Manager  Advanced  Amphibious  Assault,  Information 
Paper,  Advanced  Amphibious  Assault  Vehicle  (AAAV)  Program  1991-1992 
Independent  Technical  Risk  Assessment  Reports,  undated. 

62.  Bayard,  Richard  C.,  electronic  mail  dated  13  February  1998  to  DRPM  AAA 
personnel,  Subject:  Program  Event  History. 

63.  Direct  Reporting  Program  Manager,  Advanced  Amphibious  Assault,  AAAV 
Overview  for  General  R.  I.  Neal,  Assistant  Commandant,  U.S.  Marine  Corps,  Slide 
Presentation,  31  December  1996. 

64.  Ouimette,  Steve,  "Demonstrator  Vehicle  Sets  Speed  Record,"  Centerline,  March- 
April  1992. 

65.  United  States  Navy,  Carderock  Division,  Naval  Surface  Warfare  Center, 
Hydromechanics  Directorate  Research  and  Development  Report,  Test  and 
Evaluation  of  the  Propulsion  System  Demonstrator,  CRDKNSWC/HD-1452-01, 
June  1994. 

66.  Burt,  John  A.,  Director,  Test,  Systems  Engineering,  and  Evaluation,  Memorandum 
for  Chairman,  Conventional  Systems  Committee,  Advanced  Amphibious  Assault 
Vehicle  (AAAV)  Assessment,  undated. 

67.  Direct  Reporting  Program  Manager,  Advanced  Amphibious  Assault,  Advanced 
Amphibious  Assault  Vehicle  Acquisition  Review  Board,  Slide  Presentation,  June 
1994. 

68.  United  States  Marine  Corps,  Marine  Corps  Combat  Development  Command, 
Command,  Control,  Communications,  Computers,  and  Intelligence  (C4I) 
Requirements  for  the  AAAV(C),  ltr  C441,  16  January  1997. 

69.  Direct  Reporting  Program  Manager,  Advanced  Amphibious  Assault,  Advanced 
Amphibious  Assault  Vehicle  Integrating  IPT,  Slide  Presentation,  17  April  1997. 

70.  Direct  Reporting  Program  Manager,  Advanced  Amphibious  Assault,  Senate 
Appropriations  Committee  Brief,  Slide  Presentation,  17  July  1995. 
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71.  Direct  Reporting  Program  Manager,  Advanced  Amphibious  Assault,  Selected 
Acquisition  Report,  31  December  1995. 

72.  Direct  Reporting  Program  Manager,  Advanced  Amphibious  Assault,  Selected 
Acquisition  Report,  31  December  1996. 

73.  Direct  Reporting  Program  Manager,  Advanced  Amphibious  Assault,  Draft  Request 
for  Additional  Funding,  undated. 

74.  Report  to  the  United  States  House  of  Representatives’  National  Security  Committee 
(NSC)  on  Acceleration  of  the  Advanced  Assault  Amphibious  (AAAV)  Program,  4 
February  1997. 

75.  Direct  Reporting  Program  Manager,  Advanced  Amphibious  Assault,  Request  for 
Additional  Funding,  28  January  1997. 

76.  Direct  Reporting  Program  Manager,  Advanced  Amphibious  Assault,  Selected 
Acquisition  Report,  3 1  December  1997. 

77.  Direct  Reporting  Program  Manager,  Advanced  Amphibious  Assault,  Request  for 
Additional  Funding,  26  January  1999. 

78.  Moore,  Major  Keith,  DRPM  AAA  Test  and  Evaluation  Manager,  Personal 
Interview  with  Author,  1 9  July  1999. 

79.  Direct  Reporting  Program  Manager,  Advanced  Amphibious  Assault,  Acquisition 
Plan  No.:  DRPM  AAA  90-002,  19  March  1991. 

80.  Office  of  the  Secretary  of  Defense,  Memorandum  for  the  Assistant  Secretary  of  the 
Navy  (Research,  Development,  and  Acquisition),  Advanced  Amphibious  Assault 
Vehicle  (AAAV)  Test  and  Evaluation  Master  Plan  (TEMP),  6  March  1995. 


81.  Direct  Reporting  Program  Manager,  Advanced  Amphibious  Assault,  Acquisition 
Plan  No.:  DRPM  AAA  95-1,  28  April  1995. 

82.  Office  of  the  Secretary  of  Defense,  Memorandum  for  the  Assistant  Secretary  of  the 
Navy  (Research,  Development,  and  Acquisition),  Advanced  Amphibious  Assault 
Vehicle  (AAAV)  Test  and  Evaluation  Master  Plan  (TEMP),  20  December  1994. 

83.  Direct  Reporting  Program  Manager,  Advanced  Amphibious  Assault,  Joint 
Requirements  Oversight  Council,  Slide  Presentation,  25  April  1991. 
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84.  Johnson,  Allan  W.,  Memorandum  for  the  Assistant  Commandant,  Draft  AAA 
JROC  Memorandum  to  USD(A),  ltr  SER  RPR-1/454,  8  May  1991. 

85.  General  Dynamics  Amphibious  Systems,  Advanced  Amphibious  Assault  Vehicle 
Personnel  Vehicle  System/Subsystem  Specification,  SA-P1010000A,  Cage  Code: 
07BS6, 25  September  1997. 

86.  General  Dynamics  Amphibious  Systems,  Combined  DRPM-AAA/GDAMS  Test 
Plan,  CDRL-T005, 1  June  1999. 


87.  Dees,  Major  J.  V.,  MCCDC  AAAV  ORD  Action  Officer,  Personnel  Interview  with 
Author,  21  July  1999. 

88.  Direct  Reporting  Program  Manager  Advanced  Amphibious  Assault,  ORD 
Clarifications  to  Support  EOA,  Draft  ltr  8400  DRPMAAA/ss  Ser99.090,  6  April 
99. 

89.  Personnel  Notes,  Marine  Corps  Operational  Test  and  Evaluation  Activity,  ORD 
Clarification  Meeting,  22  July  1999. 

90.  General  Dynamics  Amphibious  Systems,  Advanced  Amphibious  Assault  Vehicle 
(AAAV)  Systems  Design  Review  (SDR),  Slide  Presentation,  14-16  May  1997. 

91.  Employee,  General  Dynamics  Land  Systems,  Personal  Interview  with  Author,  29 
September  1999. 

92.  Deegan  Phil,  General  Dynamics  Amphibious  Systems,  Test  Manager,  Personal 
Interview  with  Author,  29  September  1999. 

93.  Moore,  Major  Keith,  DRPM  AAA  Test  and  Evaluation  Manager,  Personal 
Interview  with  Author,  28  September  1999. 

94.  Robertson,  Colonel  Blake,  DRPM  AAA,  Personal  Interview  with  Author,  29 
September  1999. 

95.  Department  of  Defense,  Chairman  of  the  Joint  Chiefs  of  Staff,  CJCSI  3170.01A 
Requirements  Generation  System,  10  August  1999. 

96.  Direct  Reporting  Program  Manager  Advanced  Amphibious  Assault,  Memorandum 
for  the  Record,  Meeting  Minutes  from  23  Nov  98  Meeting  Between  DOT&E  and 
DRPM- AAA  Concerning  AAAV(C)  T&E  Plans,  undated. 
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97.  Swanek,  Anne,  DRPM  AAA  Assistant  Program  Manager  AAAV(P),  Personal 
Interview  with  Author,  29  September  1999. 

98.  General  Dynamics  Amphibious  Systems,  Virtual  Design  Database:  Risk,  “Ability 
to  complete  all  test  objectives  in  the  scheduled  amount  of  time.”,  1  August  1999. 

99.  General  Dynamics  Amphibious  Systems,  Test  Review  Issues,  Slide  Presentation, 
undated. 

100.  General  Dynamics  Amphibious  Systems,  Test  and  Evaluation  Review,  Slide 
Presentation,  8  February  1999. 

101.  Lytle,  LtCol  T.,  Operations  Officer,  Direct  Reporting  Program  Manager  Advanced 
Amphibious  Assault,  T&E  Schedule  Evaluation  for  Col.  Feigley,  20  February 
1997. 

102.  Carter,  Mr.  Tom,  and  Mr.  Bob  Jordan,  Director,  Operational  Test  and  Evaluation, 
Memorandum  for  Col.  Blake  Robertson,  Maj.  Keith  Moore,  Mr.  Joseph  Teets,  and 
Mr.  David  White,  AAAV  LFT&E  Management  Plan  Comments,  undated. 

103.  Douglas,  Brent  A.,  IDA,  Memorandum  for  Ms.  Amy  Spraitzer  and  Mr.  Tom  Carter, 
DOT&E,  Advanced  Amphibious  Assault  Vehicle  (AAAV)  Live  Fire  Test  and 
Evaluation  (LFT&E)  Management  Plan  Review,  25  September  1998. 

104.  Moore,  Major  Keith,  DRPM  AAA  Test  and  Evaluation  Manager,  Telephonic 
Interview  with  Author,  18  October  1999. 


105.  Gansler,  Hon.  Jacques  S.,  Under  Secretary  of  Defense  for  Acquisition  and 
Technology,  Speech:  Strengthening  Defense  Testing:  A  Challenge  to  the 
Community,  22  September  1999.  ~ 
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